Automatic pinning of units of memory

ABSTRACT

A determination is made that a unit of memory that provides storage for one or more in-memory configuration state registers is to be pinned in a system that includes a guest operating system and one or more hosts at one or more host levels. Based on determining that the unit of memory is to be pinned, at least one host of the one or more hosts is notified that the unit of memory is to be pinned.

BACKGROUND

One or more aspects relate, in general, to processing within a computingenvironment, and in particular, to facilitating such processing.

Computers of a computing environment include central processing units(CPUs) or processors that control processing within the computers.Behavior of a central processing unit is controlled by controlregisters. Control registers are processor registers that performparticular tasks, such as interrupt control, switching the addressingmode, paging control and/or coprocessor control, as examples.

Control registers are typically implemented as latches, such as solidstate elements directly on a processor chip. Some computers use a largenumber of control registers, as defined by the architecturalimplementation of the computers. Thus, control registers represent agrowing area of the chip.

Moreover, some computers support multi-threading in which a centralprocessing unit can execute multiple processes or threads concurrently.Each thread uses a separate set of control registers; thereby,increasing the number of control registers on a chip.

An increasing number of latch-based control registers may affectperformance, chip area and/or power consumption. For instance, controlregisters are switched during context switches, and thus, an increase inthe number of control registers, increases the cost of contextswitching. Further, with latch-based control registers, updates tocontrols occur in program order, which may also affect performance.

Different architectures may have different names for control registers.For instance, in the Power Architecture offered by InternationalBusiness Machines Corporation, Armonk, N.Y., the control registers arereferred to as special purpose register (SPRs). Other architectures mayuse other names. The use of control registers herein includes controlregisters of other names, including, for instance, SPRs, as well asothers.

SUMMARY

Shortcomings of the prior art are overcome and additional advantages areprovided through the provision of a computer program product forfacilitating processing within a computing environment. The computerprogram product includes a computer readable storage medium readable bya processing circuit and storing instructions for performing a method.The method includes, for instance, determining that a unit of memorythat provides storage for one or more in-memory configuration stateregisters is to be pinned in a system that includes a guest operatingsystem and one or more hosts at one or more host levels. Based ondetermining that the unit of memory is to be pinned, at least one hostof the one or more hosts is notified that the unit of memory is to bepinned. By pinning a memory unit that provides storage for the one ormore in-memory configuration state registers, the memory unit isprevented from moving, thus, facilitating processing and improvingperformance.

In one example, the determining is performed based on receiving a newconfiguration value for the unit of memory. The new configuration valueis, for instance, a guest address used to locate the unit of memory. Byperforming the pinning based on receiving the new configuration value,the pinning is performed automatically, facilitating processing andimproving performance.

In a further aspect, the method includes selecting a host level of theone or more host levels. The guest address is translated to a hostaddress for the selected host level, and the selected host level isnotified of the pinning. In yet a further aspect, the selecting, thetranslating and the notifying are repeated for each host level of theone or more host levels.

As one example, the notifying includes setting an indicator in a pagetable entry corresponding to the selected host level. As anotherexample, the notifying includes setting an indicator in a page tableentry corresponding to the unit of memory. In yet a further example, thenotifying includes raising an interrupt.

In a further aspect, a determination is made that the unit of memory isto be unpinned, and based on determining that the unit of memory is tobe unpinned, an indicator used to indicate pinning is reset. Thisprovides flexibility in memory management.

As one example, the unit of memory is a page of memory.

Computer-implemented methods and systems relating to one or more aspectsare also described and claimed herein. Further, services relating to oneor more aspects are also described and may be claimed herein.

Additional features and advantages are realized through the techniquesdescribed herein. Other embodiments and aspects are described in detailherein and are considered a part of the claimed aspects.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more aspects are particularly pointed out and distinctly claimedas examples in the claims at the conclusion of the specification. Theforegoing and objects, features, and advantages of one or more aspectsare apparent from the following detailed description taken inconjunction with the accompanying drawings in which:

FIG. 1A depicts one example of a computing environment to incorporateand use one or more aspects of the present invention;

FIG. 1B depicts another example of a computing environment toincorporate and use one or more aspects of the present invention;

FIG. 1C depicts further details of a processor of FIG. 1A or FIG. 1B, inaccordance with one or more aspects of the present invention;

FIG. 1D depicts further details of one example of an instructionexecution pipeline used in accordance with one or more aspects of thepresent invention;

FIG. 1E depicts further details of one example of a processor, inaccordance with an aspect of the present invention;

FIG. 2 depicts one example of in-processor configuration state registersand in-memory configuration state registers, in accordance with anaspect of the present invention;

FIG. 3 depicts one example of decode logic associated with usingin-memory configuration state registers, in accordance with an aspect ofthe present invention;

FIG. 4 depicts one example of a load configuration state registerinternal operation, in accordance with an aspect of the presentinvention;

FIG. 5 depicts one example of a store configuration state registerinternal operation, in accordance with an aspect of the presentinvention;

FIG. 6 depicts one example of using an in-memory configuration stateregister, in accordance with an aspect of the present invention;

FIG. 7 depicts another example of using an in-memory configuration stateregister, in accordance with an aspect of the present invention;

FIG. 8 depicts one example of a configuration state register writeoperation, in accordance with an aspect of the present invention;

FIG. 9 depicts one example of a configuration state register readoperation, in accordance with an aspect of the present invention;

FIG. 10 depicts one embodiment of decode logic associated with a move toor a move from configuration state register, in accordance with anaspect of the present invention;

FIG. 11 depicts further details associated with a move to configurationstate register instruction, in accordance with an aspect of the presentinvention;

FIG. 12 depicts further details of a move from configuration stateregister instruction, in accordance with an aspect of the presentinvention;

FIG. 13A depicts one embodiment of logic associated with a compositeconfiguration state register read reference, in accordance with anaspect of the present invention;

FIG. 13B depicts one embodiment of logic associated with a compositeconfiguration state register write reference, in accordance with anaspect of the present invention;

FIG. 14 depicts one example of a composite configuration state register,in accordance with an aspect of the present invention;

FIGS. 15A-15B depict one example of linear mapping of configurationstate registers, in accordance with an aspect of the present invention;

FIG. 16 depicts one example of remap flow logic for configuration stateregisters, in accordance with an aspect of the present invention;

FIG. 17A depicts one example of multiple configuration state registerstore operations;

FIG. 17B depicts one example of a bulk store configuration stateregister operation, in accordance with an aspect of the presentinvention;

FIG. 17C depicts one example of a bulk load configuration state registeroperation, in accordance with an aspect of the present invention;

FIG. 18A depicts one example of specifying an architecturalconfiguration control, in accordance with an aspect of the presentinvention;

FIG. 18B depicts another example of specifying an architecturalconfiguration control, in accordance with an aspect of the presentinvention;

FIG. 19A depicts one example of performing a context switch, inaccordance with an aspect of the present invention;

FIG. 19B depicts another example of performing a context switch, inaccordance with an aspect of the present invention;

FIG. 20 depicts one embodiment of address translation associated with amove to configuration state register operation, in accordance with anaspect of the present invention;

FIGS. 21A-21B depict examples of performing dynamic address translation,in accordance with aspects of the present invention;

FIG. 22 depicts one example of a page table entry, in accordance with anaspect of the present invention;

FIG. 23 depicts one example of particular configuration state registersbeing associated with particular contexts, in accordance with an aspectof the present invention;

FIG. 24 depicts one embodiment of providing a pinning notification to ahost system, in accordance with an aspect of the present invention;

FIG. 25 depicts one embodiment of specifying a pin operation in a pagetable entry, in accordance with an aspect of the present invention;

FIG. 26 depicts one embodiment of specifying an unpin operation in apage table entry, in accordance with an aspect of the present invention;

FIG. 27 depicts one example of combining a pin and an unpin operation inone hypervisor call, in accordance with an aspect of the presentinvention;

FIG. 28 depicts further details associated with performing a pin and anunpin operation based on a single call, in accordance with an aspect ofthe present invention;

FIGS. 29A-29C provide various examples of a data write, in accordancewith one or more aspects of the present invention;

FIGS. 30A-30C provide various examples of a data read, in accordancewith one or more aspects of the present invention;

FIGS. 31A-31B depict one embodiment of facilitating processing within acomputing environment, in accordance with an aspect of the presentinvention;

FIG. 32A depicts another example of a computing environment toincorporate and use one or more aspects of the present invention;

FIG. 32B depicts further details of the memory of FIG. 32A;

FIG. 33 depicts one embodiment of a cloud computing environment; and

FIG. 34 depicts one example of abstraction model layers.

DETAILED DESCRIPTION

In accordance with an aspect of the present invention, variousconfiguration state registers are provided in-memory rather thanin-processor. As used herein, the term “configuration state register”includes control registers; machine state registers (MSRs), such as aprogram status word (PSW) or other machine state registers; statusregisters (e.g., floating point status control register); specialpurpose register (SPRs); configuration registers; and/or other registersthat configure operations, e.g., of instructions.

Selected configuration state registers (or portions thereof in a furtheraspect) are provided in-memory, in which those registers are mapped tosystem memory and are included in the memory hierarchy which is coupledto, but separate from, the processor. The memory hierarchy includes, forinstance, load/store queues, one or more memory caches, and systemmemory (also referred to herein as main memory, central storage,storage, main storage, memory). By being in-memory, instead ofin-processor, the registers are accessed by using a memory address, andaccess requests may be re-ordered or speculatively processed. Incontrast, access requests for configuration state registers that arein-processor are not processed out-of-order or speculatively.In-processor configuration state registers are implemented as, forinstance, solid state elements (such as latches), e.g., directlyon-chip. On-chip denotes or relates to circuitry included in a singleintegrated circuit or in the same integrated circuit as a given device.

Based on the configuration state registers being stored in systemmemory, certain instructions, such as a move to configuration stateregister instruction (e.g., move to SPR (mtspr) instruction) and a movefrom configuration state register instruction (e.g., move from SPR(mfspr) instruction), are replaced by load and store instructions oroperations by instruction decode logic. The load and storeinstructions/operations that are generated are committed to storequeues, and typical load and store processing are performed.

As one example, a storage area to include the configuration stateregisters is defined by the operating system and/or a hypervisor and setaside for storing memory-based registers. In one embodiment, a physicalmemory region is architecturally specified (e.g., the first or last npages of physical memory).

In a further aspect, one or more portions of a configuration stateregister are provided in-memory, while one or more other portions of theconfiguration state register are provided in-processor. In one example,the portions provided in-memory are those used less frequently.

In yet a further aspect, a remapping of configuration state registers isprovided such that configuration state registers (or at least portionsthereof) that are typically used together, are placed in memory together(e.g., in a single cache line or adjacent cache lines) to improveprocessing performance.

Further, in another aspect, instructions or operations are provided toperform a bulk store or load of multiple configuration state registers.This is to facilitate, for instance, context switching, improvingperformance thereof.

Yet further, in one aspect, processing is facilitated and performance isimproved by defining a set of controls to identify where in memory theconfiguration state registers are stored.

In a further aspect, efficiencies are achieved during context switchingby manipulating memory pointers of in-memory configuration stateregisters. The pointers are manipulated, rather than copying the oldconfiguration data. This improves processing within the computingenvironment by increasing the speed and reducing complexity duringcontext switches.

Moreover, in another aspect, based on executing an instruction thatloads an address to be used as a base address, address translation isautomatically performed in order to avoid a potential page fault lateron in processing of the instruction.

In yet a further aspect, configuration state registers are segregated bycontext or group (e.g., hypervisor, operating system, process, thread)to facilitate processing by increasing management flexibility.

As a further aspect, an indication of automatic pinning for aninitialized memory backing state is provided.

Yet further, in another aspect, pinning of memory pages is efficientlymanaged using paravirtualized pinning calls.

Even further, in one aspect, system memory is protected against singleevent upsets.

Various aspects are described herein. Further, many variations arepossible without departing from a spirit of aspects of the presentinvention. It should be noted that, unless otherwise inconsistent, eachaspect or feature described herein and variants thereof may becombinable with any other aspect or feature.

One embodiment of a computing environment to incorporate and use one ormore aspects of the present invention is described with reference toFIG. 1A. In one example, the computing environment is based on thez/Architecture, offered by International Business Machines Corporation,Armonk, N.Y. One embodiment of the z/Architecture is described in“z/Architecture Principles of Operation,” IBM Publication No.SA22-7832-10, March 2015, which is hereby incorporated herein byreference in its entirety. Z/ARCHITECTURE is a registered trademark ofInternational Business Machines Corporation, Armonk, N.Y., USA.

In another example, the computing environment is based on the PowerArchitecture, offered by International Business Machines Corporation,Armonk, N.Y. One embodiment of the Power Architecture is described in“Power ISA™ Version 2.07B,” International Business Machines Corporation,Apr. 9, 2015, which is hereby incorporated herein by reference in itsentirety. POWER ARCHITECTURE is a registered trademark of InternationalBusiness Machines Corporation, Armonk, N.Y., USA.

The computing environment may also be based on other architectures,including, but not limited to, the Intel x86 architectures. Otherexamples also exist.

As shown in FIG. 1A, a computing environment 100 includes, for instance,a computer system 102 shown, e.g., in the form of a general-purposecomputing device. Computer system 102 may include, but is not limitedto, one or more processors or processing units 104 (e.g., centralprocessing units (CPUs)), a memory 106 (a.k.a., system memory, mainmemory, main storage, central storage or storage, as examples), and oneor more input/output (I/O) interfaces 108, coupled to one another viaone or more buses and/or other connections 110.

Bus 110 represents one or more of any of several types of busstructures, including a memory bus or memory controller, a peripheralbus, an accelerated graphics port, and a processor or local bus usingany of a variety of bus architectures. By way of example, and notlimitation, such architectures include the Industry StandardArchitecture (ISA), the Micro Channel Architecture (MCA), the EnhancedISA (EISA), the Video Electronics Standards Association (VESA) localbus, and the Peripheral Component Interconnect (PCI).

Memory 106 may include, for instance, a cache 120, such as a sharedcache, which may be coupled to local caches 122 of processors 104.Further, memory 106 may include one or more programs or applications130, an operating system 132, and one or more computer readable programinstructions 134. Computer readable program instructions 134 may beconfigured to carry out functions of embodiments of aspects of theinvention.

Computer system 102 may also communicate via, e.g., I/O interfaces 108with one or more external devices 140, one or more network interfaces142, and/or one or more data storage devices 144. Example externaldevices include a user terminal, a tape drive, a pointing device, adisplay, etc. Network interface 142 enables computer system 102 tocommunicate with one or more networks, such as a local area network(LAN), a general wide area network (WAN), and/or a public network (e.g.,the Internet), providing communication with other computing devices orsystems.

Data storage device 144 may store one or more programs 146, one or morecomputer readable program instructions 148, and/or data, etc. Thecomputer readable program instructions may be configured to carry outfunctions of embodiments of aspects of the invention.

Computer system 102 may include and/or be coupled toremovable/non-removable, volatile/non-volatile computer system storagemedia. For example, it may include and/or be coupled to a non-removable,non-volatile magnetic media (typically called a “hard drive”), amagnetic disk drive for reading from and writing to a removable,non-volatile magnetic disk (e.g., a “floppy disk”), and/or an opticaldisk drive for reading from or writing to a removable, non-volatileoptical disk, such as a CD-ROM, DVD-ROM or other optical media. Itshould be understood that other hardware and/or software componentscould be used in conjunction with computer system 102. Examples,include, but are not limited to: microcode, device drivers, redundantprocessing units, external disk drive arrays, RAID systems, tape drives,and data archival storage systems, etc.

Computer system 102 may be operational with numerous other generalpurpose or special purpose computing system environments orconfigurations. Examples of well-known computing systems, environments,and/or configurations that may be suitable for use with computer system102 include, but are not limited to, personal computer (PC) systems,server computer systems, thin clients, thick clients, handheld or laptopdevices, multiprocessor systems, microprocessor-based systems, set topboxes, programmable consumer electronics, network PCs, minicomputersystems, mainframe computer systems, and distributed cloud computingenvironments that include any of the above systems or devices, and thelike.

In another embodiment, the computing environment supports virtualmachines. One example of such an environment is described with referenceto FIG. 1B. In one example, a computing environment 161 includes acentral processor complex (CPC) 163 providing virtual machine support.CPC 163 is coupled to one or more input/output (I/O) devices 167 via oneor more control units 169. Central processor complex 163 includes, forinstance, a memory 165 (a.k.a., system memory, main memory, mainstorage, central storage, storage) coupled to one or more processors(a.k.a., central processing units (CPUs)) 171, and an input/outputsubsystem 173, each of which is described below.

Memory 165 includes, for example, one or more virtual machines 175, avirtual machine manager, such as a hypervisor 177, that manages thevirtual machines, and processor firmware 179. One example of hypervisor177 is z/VM, offered by International Business Machines Corporation,Armonk, N.Y. The hypervisor is sometimes referred to as a host. Further,as used herein, firmware includes, e.g., the microcode of the processor.It includes, for instance, the hardware-level instructions and/or datastructures used in implementation of higher level machine code. In oneembodiment, it includes, for instance, proprietary code that istypically delivered as microcode that includes trusted software ormicrocode specific to the underlying hardware and controls operatingsystem access to the system hardware.

The virtual machine support of the CPC provides the ability to operatelarge numbers of virtual machines 175, each capable of operating withdifferent programs 185 and running a guest operating system 183, such asLinux. Each virtual machine 175 is capable of functioning as a separatesystem. That is, each virtual machine can be independently reset, run aguest operating system, and operate with different programs. Anoperating system or application program running in a virtual machineappears to have access to a full and complete system, but in reality,only a portion of it is available.

Memory 165 is coupled to processors (e.g., CPUs) 171, which are physicalprocessor resources assignable to virtual machines. For instance,virtual machine 175 includes one or more logical processors, each ofwhich represents all or a share of a physical processor resource 171that may be dynamically allocated to the virtual machine.

Further, memory 165 is coupled to an I/O subsystem 173. Input/outputsubsystem 173 directs the flow of information between input/outputcontrol units 169 and devices 167 and main storage 165. It is coupled tothe central processing complex, in that it can be a part of the centralprocessing complex or separate therefrom.

Further details regarding one example of a processor, such as processor104 (or processor 171), are described with reference to FIG. 1C. Aprocessor, such as processor 104 (or processor 171), includes aplurality of functional components used to execute instructions. Thesefunctional components include, for instance, an instruction fetchcomponent 150 to fetch instructions to be executed; an instructiondecode unit 152 to decode the fetched instructions and to obtainoperands of the decoded instructions; instruction execution components154 to execute the decoded instructions; a memory access component 156to access memory for instruction execution, if necessary; and a writeback component 160 to provide the results of the executed instructions.One or more of these components may, in accordance with an aspect of thepresent invention, be used to execute one or more instructions and/oroperations associated with memory-based configuration state registerprocessing 166.

Processor 104 (or processor 171) also includes, in one embodiment, oneor more registers 168 to be used by one or more of the functionalcomponents. Processor 104 (or processor 171) may include additional,fewer and/or other components than the examples provided herein.

Further details regarding an execution pipeline of a processor, such asprocessor 104 or processor 171, are described with reference to FIG. 1D.Although various processing stages of the pipeline are depicted anddescribed herein, it will be understood that additional, fewer and/orother stages may be used without departing from the spirit of aspects ofthe invention.

Referring to FIG. 1D, in one embodiment, an instruction is fetched 170from an instruction queue, and branch prediction 172 and/or decoding 174of the instruction may be performed. The decoded instruction may beadded to a group of instructions 176 to be processed together. Thegrouped instructions are provided to a mapper 178 that determines anydependencies, assigns resources and dispatches the group ofinstructions/operations to the appropriate issue queues. There are oneor more issue queues for the different types of execution units,including, as examples, branch, load/store, floating point, fixed point,vector, etc. During an issue stage 180, an instruction/operation isissued to the appropriate execution unit. Any registers are read 182 toretrieve its sources, and the instruction/operation executes during anexecute stage 184. As indicated, the execution may be for a branch, aload (LD) or a store (ST), a fixed point operation (FX), a floatingpoint operation (FP), or a vector operation (VX), as examples. Anyresults are written to the appropriate register(s) during a write backstage 186. Subsequently, the instruction completes 188. If there is aninterruption or flush 190, processing may return to instruction fetch170.

Further, in one example, coupled to the decode unit is a registerrenaming unit 192, which may be used in the saving/restoring ofregisters.

Additional details regarding a processor are described with reference toFIG. 1E. In one example, a processor, such as processor 104 (orprocessor 171), is a pipelined processor that may include predictionhardware, registers, caches, decoders, an instruction sequencing unit,and instruction execution units, as examples. The prediction hardwareincludes, for instance, a local branch history table (BHT) 105 a, aglobal branch history table (BHT) 105 b, and a global selector 105 c.The prediction hardware is accessed through an instruction fetch addressregister (IFAR) 107, which has the address for the next instructionfetch.

The same address is also provided to an instruction cache 109, which mayfetch a plurality of instructions referred to as a “fetch group”.Associated with instruction cache 109 is a directory 111.

The cache and prediction hardware are accessed at approximately the sametime with the same address. If the prediction hardware has predictioninformation available for an instruction in the fetch group, thatprediction is forwarded to an instruction sequencing unit (ISU) 113,which, in turn, issues instructions to execution units for execution.The prediction may be used to update IFAR 107 in conjunction with branchtarget calculation 115 and branch target prediction hardware (such as alink register prediction stack 117 a and a count register stack 117 b).If no prediction information is available, but one or more instructiondecoders 119 find a branch instruction in the fetch group, a predictionis created for that fetch group. Predicted branches are stored in theprediction hardware, such as in a branch information queue (BIQ) 125,and forwarded to ISU 113.

A branch execution unit (BRU) 121 operates in response to instructionsissued to it by ISU 113. BRU 121 has read access to a condition register(CR) file 123. Branch execution unit 121 further has access toinformation stored by the branch scan logic in branch information queue125 to determine the success of a branch prediction, and is operativelycoupled to instruction fetch address register(s) (IFAR) 107corresponding to the one or more threads supported by themicroprocessor. In accordance with at least one embodiment, BIQ entriesare associated with, and identified by an identifier, e.g., by a branchtag, BTAG. When a branch associated with a BIQ entry is completed, it isso marked. BIQ entries are maintained in a queue, and the oldest queueentries are de-allocated sequentially when they are marked as containinginformation associated with a completed branch. BRU 121 is furtheroperatively coupled to cause a predictor update when BRU 121 discovers abranch misprediction.

When the instruction is executed, BRU 121 detects if the prediction iswrong. If so, the prediction is to be updated. For this purpose, theprocessor also includes predictor update logic 127. Predictor updatelogic 127 is responsive to an update indication from branch executionunit 121 and configured to update array entries in one or more of thelocal BHT 105 a, global BHT 105 b, and global selector 105 c. Thepredictor hardware 105 a, 105 b, and 105 c may have write ports distinctfrom the read ports used by the instruction fetch and predictionoperation, or a single read/write port may be shared. Predictor updatelogic 127 may further be operatively coupled to link stack 117 a andcount register stack 117 b.

Referring now to condition register file (CRF) 123, CRF 123 isread-accessible by BRU 121 and can be written to by the execution units,including but not limited to, a fixed point unit (FXU) 141, a floatingpoint unit (FPU) 143, and a vector multimedia extension unit (VMXU) 145.A condition register logic execution unit (CRL execution) 147 (alsoreferred to as the CRU), and special purpose register (SPR) handlinglogic 149 have read and write access to condition register file (CRF)123. CRU 147 performs logical operations on the condition registersstored in CRF file 123. FXU 141 is able to perform write updates to CRF123.

Processor 104 (or processor 171) further includes, a load/store unit151, and various multiplexors 153 and buffers 155, as well as addresstranslation tables 157, and other circuitry.

Further details regarding various registers used by a processor 200,such as processor 104 or processor 171, are described with reference toFIG. 2. As shown, processor 200 includes a plurality of in-processorconfiguration state registers (CSRs) 202. As examples, the in-processorconfiguration state registers include a link register (LR), a counterregister (CTR), a machine state register (MSR), a floating point statuscontrol register (FPSCR), a next instruction address (NIA) register, andone or more integer exception registers (XER) registers. Further, inaccordance with an aspect of the present invention, system memory 206coupled to processor 200 includes one or more in-memory configurationstate registers 208. As examples, the in-memory configuration stateregisters include event based branch return registers (EBBRR), eventbased branch registers (EBB), state restoration registers (SRRs); aninteger exception register (XER); and a vector register save register(VRSAVE). In one example, the in-memory configuration state registers208 are stored in an in-memory configuration state register area 210 ofsystem memory 206.

A configuration state register that is accessed frequently (e.g.,several accesses in a row) may be moved to a cache hierarchy 212 coupledto processor 200 and system memory 206.

In accordance with one aspect, based on one or more configuration stateregisters being moved or placed in-memory, in-processor accesses tothose configuration state registers are replaced with accesses tomemory. One example of decode logic that determines the type of accessis described with reference to FIG. 3. This processing is performed by,e.g., the decode unit and/or another unit of the processor.

Referring to FIG. 3, initially, an instruction is received, STEP 300. Adetermination is made as to whether the instruction is a move toconfiguration state register instruction, such as a Move to SPR (mtspr)instruction, INQUIRY 302. If the instruction is a move to configurationstate register instruction, then a further determination is made as towhether the configuration state register indicated in the instruction isan in-memory configuration state register, INQUIRY 304. If not, thenconventional handling of the move to configuration state registerinstruction is performed, STEP 306. However, if the configuration stateregister is an in-memory configuration state register, then a storeconfiguration state register internal operation is generated to storethe configuration state register in memory (e.g., store the new contentsof the configuration state register in memory), STEP 308.

Returning to INQUIRY 302, if the received instruction is not a move toconfiguration state register instruction, then a further determinationis made as to whether the instruction is a move from configuration stateregister instruction, such as a Move from SPR (mfspr) instruction,INQUIRY 312. If the instruction is a move from configuration stateregister instruction, then a determination is made as to whether theconfiguration state register indicated in the instruction is in-memory,INQUIRY 314. If not, then conventional move from configuration stateregister handling is performed, STEP 316. Otherwise, a loadconfiguration state register internal operation is generated to obtainthe contents of the register from memory, STEP 318.

Returning to INQUIRY 312, if the received instruction is not a move toor move from configuration state register instruction, then yet afurther determination may be performed to determine whether the receivedinstruction is another instruction that uses a configuration stateregister, INQUIRY 322. If so, then a read and/or write internaloperation may be generated depending on the function being performed bythe instruction, STEP 324. Otherwise, processing continues to STEP 332,in which conventional instruction decode processing is performed.

In other aspects of the present invention, internal operations to loadconfiguration state register and store configuration state registervalues are performed in conjunction with the performing of processoroperations not corresponding to instructions, e.g., in response toentering an exception handler responsive to receiving an interruptrequest.

Further details regarding a load configuration state register internaloperation are described with reference to FIG. 4. This processing isperformed by a processor. Referring to FIG. 4, initially, a memory baseaddress (base) is obtained from a register or location (e.g., a baseregister, such as a thread control base register (TCBR)) that containsan address of a memory unit (e.g., memory page) that is the base addressof the memory that includes the configuration state registers, STEP 400.Additionally, a register number indicated in the operation is obtained,STEP 402. That register number is mapped to an offset in memory, STEP404. For instance, each configuration state register number (or otheridentification in another embodiment) is mapped to a particular locationin memory. That location, is a certain amount (e.g., offset) from thebase address. Then, a load from an address (base address plus offset) isperformed, STEP 406, and the loaded value is returned, STEP 408.

As used herein, base refers to a base address of the memory thatincludes the in-memory configuration state registers, and base registerrefers to a register that includes the base. One example of a baseregister is a thread control base register (TCBR), but other contexts(e.g., operating system, etc.) may use other base registers.

Further details regarding a store configuration state register internaloperation are described with reference to FIG. 5. This processing isperformed by a processor. Referring to FIG. 5, initially, a memory baseaddress (base) is obtained, e.g., from a base register (e.g., fromTCBR), STEP 500, as well as, the register number indicated in theoperation, STEP 502. The register number is mapped to an offset inmemory, STEP 504, and a store operand (e.g., the contents of theregister) is stored at an address specified by the base address plusoffset, STEP 506.

As indicated above, instructions other than a move from or a move toconfiguration state register instruction may use a configuration stateregister. Thus, processing associated with one of these instructions isdescribed with reference to FIG. 6. This processing is performed by aprocessor. Referring to FIG. 6, in this embodiment, aninstruction/operation is obtained that includes a configuration stateregister read reference, STEP 600. Based thereon, a memory base address(base) for the configuration state register indicated in theinstruction/operation is obtained, e.g., from a base register (e.g.,from TCBR), STEP 602, as well as the register number indicated in theinstruction/operation, STEP 604. The register number is mapped to anoffset in memory, STEP 606, and the contents from the address specifiedby base plus offset are loaded into a temporary register, STEP 608. Thetemporary register is then used, STEP 610.

Similar processing is performed for a configuration state register writereference, as described with reference to FIG. 7. This processing isperformed by a processor. Referring to FIG. 7, in one example, aconfiguration state register write reference is obtained, STEP 700.Based thereon, a memory base address (base) is obtained, e.g., from abase register (e.g., from TCBR) for the configuration state registerindicated in the instruction/operation, STEP 702, in addition to theregister number specified in the instruction/operation, STEP 704. Theregister number is mapped to an offset, STEP 706, and the contentsincluded in the write reference (e.g., in a temporary register) arestored at the address specified at base plus offset, STEP 708.

Further details regarding an operational implementation view of aconfiguration state register write operation (such as a move toconfiguration state register (e.g., mtspr)) are described with referenceto FIG. 8. This processing is performed by a processor. Referring toFIG. 8, in one example, the register number specified in the operationis translated, STEP 800. For instance, the memory address correspondingto or mapped to the register number (or other indication) is determined(e.g., by using a look-up table or calculated). Further, a store queueentry is allocated, STEP 802, and the address corresponding to thesubject configuration state register is stored into the store queueentry, STEP 804. Moreover, the contents (e.g., data value(s))corresponding to the data written to the subject configuration stateregister is written into the store queue entry, STEP 806. In oneexample, STEPS 804 and 806 may be performed out-of-order.

The store queue entry is monitored for reads to the indicated address(e.g., bypass from store queue), STEP 808. Further, the store queueentry may be flushed based on a mispeculation event, which, in oneexample, can occur up to an architectural in-order point, STEP 810.

The contents (e.g., data values) are written to an address in the memoryhierarchy, e.g., a first level cache, STEP 812. The data from the firstlevel cache is provided based on a read request, STEP 814. Further,based on a cache replacement policy, data is evicted from the firstlevel cache to one or more next levels of the cache hierarchy, STEP 816.Data from one or more next levels of the cache hierarchy are providedbased on a read request, STEP 818. Based on the cache replacementpolicy, data from the cache levels is evicted to system memory, e.g.,DRAM, STEP 820. The data from the system memory is provided based on aread request, STEP 822.

Further details regarding an operational implementation view of aconfiguration state register read operation are described with referenceto FIG. 9. This processing is performed by a processor. Referring toFIG. 9, in one example, the register number specified by the readoperation is translated to a corresponding memory address, STEP 900. Aload sequence number used to indicate a position in a load queue used totrack load requests and a load tag that tracks dependencies areobtained, STEP 902. A test for the presence of data at the addresscorresponding to the configuration state register being read in thestore queue is performed, STEP 904 (i.e., is the data to be read frommemory in the store queue). If the data for the configuration stateregister to be read is found in the store queue, INQUIRY 906, then thevalue is obtained from the store queue, STEP 908, and processing iscomplete.

Returning to INQUIRY 906, if the data for the configuration stateregister to be read is not found in the store queue, then a furtherdetermination is made as to whether the data for the configuration stateregister to be read is found in the first level cache, INQUIRY 910. Ifso, then the value is obtained from the first level cache, STEP 912, andprocessing is complete.

Returning to INQUIRY 910, however, if the data is not found in the firstlevel cache, then a further determination is made as to whether the datafor the configuration state register to be read is found in one or morenext level caches, INQUIRY 914. If the data is found in one or more nextlevel caches, then the data is obtained from a next level cache, STEP916, and processing is complete.

Returning to INQUIRY 914, if the data is not in one or more next levelcaches, then a read request is issued to the load queue to retrieve thedata from system memory, STEP 918. The data corresponding to theconfiguration state register is obtained when the load from systemmemory completes, STEP 920.

In accordance with an aspect of the present invention, the allocation ofin-memory memory units (e.g., pages) is performed in order to providesoftware compatibility. For instance, the allocation is performed byfirmware for a processor to be able to execute legacy hypervisors, andhypervisors to be able to execute legacy operating systems, and soforth.

In one example, upon initial boot of the system, firmware allocatesin-memory pages for the in-memory configuration state registers, infirmware-owned memory. In one example, if a hypervisor is unaware of thein-memory configuration state registers, then the firmware-owned pagesare used throughout the entire execution of the system without anyfurther software reference to a base register, e.g., TCBR, etc.

As such, the hypervisor will simply perform context switches by readingthe context using, e.g., a move from configuration state register, e.g.,mfspr, and reloading the context with, e.g., a move to configurationstate register, e.g., mtspr. This offers significant design simplicityand performance advantages within the computer system.

Further, in one example, when a hypervisor is aware of memory-backedpages, it may configure each new partition to have a set of backingpages. Further, if an operating system is unaware of in-memoryconfiguration state registers, then the hypervisor owned page(s) is usedthroughout the entire execution of the system without any furthersoftware reference to, e.g., a base register, e.g., TCBR, etc. If thehypervisor is unaware as well, then the operating system will usefirmware-owned pages.

As such, the operating system will simply perform context switches byreading the context using, e.g., a move from configuration stateregister, such as, e.g., mfspr, and reloading the context with, e.g., amove to configuration state register, such as, e.g., mtspr. This offerssignificant design simplicity and performance advantages, facilitatingprocessing within the computer system.

As described herein, in accordance with one or more aspects, selectedconfiguration state registers are stored in system memory. Thus, move toand from configuration state registers are replaced by load and storeinstructions by instruction decode logic. Loads and stores that are sogenerated are committed to store queues, and normal load and storeprocessing are performed. In one example, the configuration stateregisters that are not constantly needed (e.g., those other thanregisters, such as the program counter (PC), data and address breakpoint registers, PSW, floating point control, etc.) are those stored inmemory.

As an example, the storage area is defined by the operating system andhypervisor and set aside for storing memory-based registers. In oneembodiment, a physical memory region is architecturally specified (e.g.,the first or last n pages of physical memory).

In at least one embodiment, in-memory configuration state registers aremapped to normal cacheable memory. When a configuration state registeris to be updated, it is stored into a store queue. The store queue isnot only a queuing mechanism, but effectively provides a way to renamelocations for storage in order to enable speculative execution of memoryaccesses. Multiple versions of speculative values for an address can bein the store queue (in addition to an authoritative, architected valueat an architectural in-order point, which is in cache or system memory).The cache entries may be updated out-of-order, once they have beenallocated. Also stores may be undone by flushing entries from the storequeue.

Consequently, an in-memory configuration state register can be updatedby using the store queue and read back out-of-order with no performancecost, where an in-core latch based configuration state register forcestwo serialization and in-order access cost penalties becauseimplementing means for speculative execution of in-processorconfiguration state registers are most often prohibitively expensive.

Further, when a value is not in the store queue, a read of the value canbe done more efficiently than from a latch because frequently usedmemory controls (e.g., in-memory configuration state registers) will befound in the cache and may be available in as little as 2-3 cycles (timeto access a first level cache), much faster than the special purposelogic needed to access a latch based configuration state register in aprocessor.

In one embodiment, when a page is allocated to hold configuration stateregisters, the architecture disallows access to the page using memoryoperands. This avoids an interlock between a memory operation and movefrom/move to configuration state register instructions.

In accordance with a further aspect of the present invention, one ormore portions of a configuration state register are provided in-memory,while one or more other portions of the configuration state register areprovided in-processor. For instance, a configuration state register mayhave a plurality of portions (e.g., fields) and one or more of thoseportions that are, for instance, frequently accessed, may remainin-processor and the remaining portions that are, for instance,infrequently used, may be moved to memory. This is described in furtherdetail with reference to FIGS. 10-14.

Initially, referring to FIG. 10, the decode unit of a processor (oranother component) receives an instruction, STEP 1000. A determinationis made by the decode unit (or another component) as to whether thatinstruction is a move to configuration state register instruction(mtcsr), such as to a Move to SPR (mtspr) instruction, INQUIRY 1002. Ifit is a move to configuration state register instruction, then thatinstruction is handled, STEP 1004, as described below.

Returning to INQUIRY 1002, if the instruction is not a move toconfiguration state register instruction, then a further determinationis made as to whether the instruction is a move from configuration stateregister instruction (mfcsr), such as a Move from SPR (mfspr)instruction, INQUIRY 1006. If it is a move from configuration stateregister instruction, then that instruction is handled, as describedbelow, STEP 1008. Returning to INQUIRY 1006, if the instruction isneither a move to or a move from configuration state registerinstruction, then conventional instruction decode is performed, STEP1010.

In a further embodiment, other inquiries may be made as to whether it isanother instruction that uses a configuration state register, and if so,those instructions may be handled, as appropriate, examples of which aredescribed herein. In yet a further embodiment, processor operations notcorresponding to instructions (e.g., initiating an exception handlingsequence) may be similarly performed.

Further details regarding handling a move to configuration stateregister instruction are described with reference to FIG. 11. In oneexample, the configuration state register may be a special purposeregister (SPR), and the instruction is a Move to SPR (mtspr)instruction. However, this is only one example. Other configurationstate registers may be processed similarly. This logic is performed by aprocessor, such as, for instance, the decode unit of the processor. Inother examples, one or more other components perform this logic.

Referring to FIG. 11, based on obtaining (e.g., receiving, provided,selecting, etc.) a move to configuration state register instruction,such as an mtspr instruction, a determination is made as to whether aleast a portion of the configuration state register (CSR) specified bythe instruction is in-memory, INQUIRY 1100. If not, then conventionalprocessing of the move to configuration state register instruction(e.g., mtspr) is performed, STEP 1102.

Returning to INQUIRY 1100, if at least a portion of the configurationstate register is in-memory, then a further determination is made as towhether the entire configuration state register is in-memory, INQUIRY1104. If the entire configuration state register is in-memory, then astore configuration state register internal operation is generated, STEP1106. An example of processing associated with this internal operationis described with reference to FIG. 5.

Returning to INQUIRY 1104, if only one or more portions of theconfiguration state register are in-memory, then one or more storeconfiguration state register operations are generated for the one ormore in-memory configuration state register portions, STEP 1110.Further, updated internal operations are generated for the one or morein-core configuration state register portions, STEP 1112. The updatedinternal operations may be one or more instructions, a state machine orother that performs the operation of copying the contents of one or moregeneral purpose registers including data for the specified in-coreportions to the appropriate portion(s) of the in-core configurationstate register. Processing is complete.

Further details regarding processing associated with handling a movefrom configuration state register instruction are described withreference to FIG. 12. In one example, the configuration state registermay be a special purpose register (SPR), and the instruction is a Movefrom SPR (mfspr) instruction. However, this is only one example. Otherconfiguration state registers may be processed similarly. This logic isperformed by a processor, such as, for instance, the decode unit of theprocessor. In other examples, one or more other components perform thislogic.

Referring to FIG. 12, based on obtaining (e.g., receiving, providing,selecting, etc.) a move from configuration state register instruction,such as an mfspr instruction, a determination is made as to whether atleast a portion of the configuration state register is in-memory. Ifnot, then conventional processing is performed for the move fromconfiguration state register instruction, STEP 1202.

Returning to INQUIRY 1200, if at least a portion of the configurationstate register is in-memory, then a further determination is made as towhether the entire configuration state register is in-memory, INQUIRY1204. If the entire configuration state register is in-memory, then aload configuration state register internal operation is generated, STEP1206. An example of processing associated with this operation isdescribed with reference to FIG. 4.

Returning to INQUIRY 1204, if only one or more portions of theconfiguration state register are in-memory, then one or more loadconfiguration state register internal operations are generated for oneor more of the in-memory configuration state register portions, STEP1210. Further, one or more read internal operations are generated forthe one or more in-core configuration state register portions, STEP1212.

Additionally, in one embodiment, one or more internal operations aregenerated to combine the in-memory and in-core portions into anarchitecturally defined configuration state register image, STEP 1214.This may include using, for instance, an Insert Under Mask instruction,or OR, AND, and/or NOT logic circuits, as described further below.

Further details regarding the use of a composite configuration stateregister in which one or more portions are in-processor and one or moreportions are in-memory are described with reference to FIG. 13A, inwhich a read reference is described. This logic is performed by aprocessor, such as, for instance, the decode unit of the processor. Inother examples, one or more other components perform this logic.

Referring to FIG. 13A, based on a composite configuration state registerread reference 1300, a determination is made as to whether a particularportion (also referred to as a component; e.g., a field) being accessedis in-memory or in-processor, INQUIRY 1310. If it is in-processor, thenthe in-processor component is accessed, STEP 1320, and processingcontinues to INQUIRY 1350, described below. However, if the particularcomponent is in-memory, INQUIRY 1310, then processing is performed, asdescribed with reference to FIG. 6. For instance, the memory baseaddress (base) is obtained, STEP 1330, as well as a register numberindicated in the instruction referencing the composite configurationstate register, STEP 1332. The register number is mapped to an offset,STEP 1334, and a load is performed from the address (base+offset) to atemporary register, STEP 1336. The temporary register is then used, STEP1338. Thereafter, or after STEP 1320, a determination is made as towhether another component of the composite configuration register is tobe accessed, INQUIRY 1350. If so, then processing continues with INQUIRY1310. Otherwise, processing is complete.

Further details regarding the use of a composite configuration stateregister in which one or more portions are in-processor and one or moreportions are in-memory are described with reference to FIG. 13B, inwhich a write reference is described. This logic is performed by aprocessor, such as, for instance, the decode unit of the processor. Inother examples, one or more other components perform this logic.

Referring to FIG. 13B, based on a composite configuration state registerwrite reference 1360, a determination is made as to whether a particularportion (also referred to as a component; e.g., a field) being accessedis in-memory or in-processor, INQUIRY 1370. If it is in-processor, thenthe in-processor component is accessed, STEP 1390, and processingcontinues to INQUIRY 1388, described below. However, if the particularcomponent is in-memory, INQUIRY 1370, then processing is performed, asdescribed with reference to FIG. 7. For instance, the memory baseaddress (base) is obtained, STEP 1380, as well as a register numberindicated in the instruction referencing the composite configurationstate register, STEP 1382. The register number is mapped to an offset,STEP 1384, and a store is performed to an address defined bybase+offset, STEP 1386. Thereafter, or after STEP 1390, a determinationis made as to whether another component of the composite configurationregister is to be accessed, INQUIRY 1388. If so, then processingcontinues with INQUIRY 1370. Otherwise, processing is complete.

One example of a composite configuration state register is depicted inFIG. 14. As shown, in this example, a composite configuration stateregister 1400 is special purpose register (SPR) 1, which corresponds toan integer exception register (XER). This register includes a pluralityof fields 1402. In one example, one or more of the fields arein-processor fields 1404, and another field 1406 is an in-memory field.In this particular example, Xerf0, 1, 2 (i.e., fields 0, 1, and 2 ofXER) are renamed in-processor to SO (summary overflow), OV (overflow)and CA (carry); and Xerf3 (field 3 of XER), which is not renamed in thisexample, is a byte count field in-memory. With this configuration, thefollowing IOP sequences may be generated and used to perform a mtspr anda mfspr, respectively, for a composite configuration state register:

mtspr_xer mtxerf2 mtxerf0 mtxerf1 stxerf3

With the above, the mtspr for the XER register includes: a move to field2 of XER (mtxerf2), in which contents of a general purpose register arecopied to XER field 2; a move to field 0 of XER (mtxerf0), in whichcontents of a general purpose register are copied to XER field 0; and amove to field 1 of XER (mtxerf1), in which contents of a general purposeregister are copied to XER field 1. It also includes a store to field 3of XER (stxerf3), which is performed by a store operation, since field 3is in memory.

mfspr_xer mfxerf2 mfxerf0 or ldxerf3 or mfxerf1 or

For the move from XER, each of the fields is read, either fromin-processor or in-memory and those fields are combined by, e.g., an ORoperation. For example, the contents of field 2 and field 0 are read andan OR operation is performed to provide a result 1; then, the contentsof field 3 are read (e.g., using a load, such as, for example, a loadxerf3 internal operation, ldxerf3, since field 3 is in-memory) and OR'dwith result 1 to produce result 2. Further, the contents of field 1 areread and OR'd with result 2 to provide a final result, which is an imageof XER with its fields, regardless of whether in-processor or in-memory.

As described herein, in accordance with an aspect of the presentinvention, a move from configuration state register instructiongenerates a sequence of moves from the in-processor portions, and a readfor the portion stored in-memory. The contents of the read in-memory andin-processor portions are collated, e.g., using a sequence of, e.g., ORinstructions. Further, a move to configuration state registerinstruction generates a sequence of moves to the in-processor portions,and a store for the portion stored in-memory.

In one aspect, when memory is assigned to configuration state registers,the offsets are architecturally (e.g., defined and externally visible)or micro-architecturally (defined but not externally visible) specified.For instance, an offset may be derived directly from the configurationstate register number (or other indication).

As one example mapping, each configuration state register is mapped tothe corresponding offset (in doublewords), i.e., base*configurationstate register #, in which configuration state register 1 is at a firstlocation; configuration state register 2 is at the first location plus adefined number of bytes (e.g., 8), and so forth.

However, configuration state register numbers are non-contiguous,wasting memory, and cache efficiency. Thus, in another embodiment inaccordance with an aspect of the present invention, the configurationstate register number is not used to directly derive an offset into amemory page, rather configuration state registers are allocated offsetsbased on functional affinity. Thus, configuration state registers thatare used together in common operations are allocated to the same oradjacent cache lines, to enhance cache locality. For example, EBBhandling uses the following registers: e.g., EBBHR, EBBRR, BESCR, andTAR. TAR is not contiguous with the others. However, they all are to beallocated to memory, so that they end up in the same cache line or anadjacent cache line.

One example of a linear mapping is depicted in FIGS. 15A-15B. As shown,in one example, a linear mapping 1500 is sparse. For instance, in oneexample, 8 KB (2 pages) is used, even though fewer than 1K ofconfiguration state registers is mapped. Further, jointly usedconfiguration state registers, such as EBBHR, EBBRR, BESCR, and TAR, arenot contiguous. Additionally, groups of configuration state registersare not on an alignment boundary to ensure they are in the same cacheline (e.g., 779 MMCR0; 780 STAR; 781 SDAR; 782 MMCR1). Yet further, someconfiguration state registers may refer to the same register; e.g.,different access permissions, subfields, etc. This is an inefficient useof cache. There is a lack of prefetch (to ensure each activity onlysuffers one cache miss); and an overly large cache foot print (resultingin an increased working set which reduces hit rate). Thus, in accordancewith an aspect of the present invention, configuration state registersare not stored at, e.g., base+(idx*8). Rather, they are stored at, forinstance, base+remap[idx].

This remap ensures groups are adjacent in order to share a cache line;it eliminates/reduces sparsity, providing a more efficient cache use;and handles multiple names. As one example, the remapping is static andis performed at processor design and provided in a data structure, suchas a table, or by computation of a defined equation. As another example,the remapping is dynamic and determined by use. For instance, iftracking of registers shows that registers of a set of registers areused together, then those registers are grouped and placed adjacent toone another. Other possibilities exist.

Further details of remapping are described with reference to FIG. 16.This processing is performed by a processor. In one example, aconfiguration state register number is obtained by the processor, STEP1600. Based on the configuration state register number, an indexposition (a.k.a., an offset) into the memory unit (e.g., page) isdetermined, STEP 1602. This may be determined by a table look-up or bycomputation. The index position is returned to the requester (e.g., aninternal operation), STEP 1604.

In a further example, a circuit that includes mapping logic is used. Aconfiguration state register number is input to the mapping logic, andthe output is a page index.

As described above, in one embodiment, the configuration state registernumbers, as defined in the architecture, are remapped such that thoseconfiguration state registers that are used together are placed close toone another in order to provide a more efficient cache. This reduces thenumber of cache lines used for the configuration state registers andcompetes less with other programs for use of the cache. Further, itensures that once a cache line is loaded with a particular configurationstate register and the penalty is paid for a cache miss for that value,other configuration state registers that may also be used in conjunctionwith that configuration state register will hit in the cache andconsequently not involve other cache miss penalties.

In a further embodiment, a sync_o_csr instruction writes a version stampinto a page when different offset assignments are possible. A versionstamp may be used to adjust offsets when migrating a partition betweendifferent hosts; and/or to adjust offsets either directly in hardware(e.g., when a sync_i_csr may read an offset version number for apartition) or in software.

In accordance with another aspect of the present invention, a capabilityis provided to perform a bulk operation to store or load multipleconfiguration state registers. An individual load or store of aconfiguration state register is expensive, since each read or write isto be performed in-order, and is to complete before the next instructionis started. Further, correct exception/error sequencing is to beensured, since a lack of renaming for in-processor configuration stateregisters disallows rollback. However, in accordance with one or moreaspects of the present invention, bulk configuration state register loadto and store from memory units, such as pages, are provided.

For instance, a store configuration state register instruction oroperation (e.g., ST_CSR) is used to store multiple in-processorconfiguration state registers in-memory (i.e., store the contents of theconfiguration state registers associated with the current context (e.g.,application, thread, etc.) in select memory locations defined for theparticular configuration state registers); and a load configurationstate register instruction or operation (e.g., LD_CSR) is used to loadconfiguration state registers stored in-memory back to in-processor(i.e., load the contents of the configuration state registers associatedwith the current context from memory back into the processor).

In-processor (also referred to as non-memory backed) configuration stateregisters that may be stored in memory are assigned locations in memory,e.g., in backing memory units (e.g., pages). In one example, the memorylocations and/or units are well-defined, pre-defined locations/units,and therefore, the instructions need not have an operand to specify thelocations/units. In another embodiment, the specific locations/units maybe specified as an operand of the instruction. Further in anotherembodiment, each instruction may include an operand to indicate specificregisters to be stored/loaded. Other variations are also possible.

Further, pages are just one example of memory units. Other units arepossible. Also, although a page is typically 4 KB, in other embodiments,it may be other sizes. Many possibilities exist.

In addition to the ST_CSR and LD_CSR instructions described above,another instruction that may be used, in accordance with an aspect ofthe present invention, is mtspr TCBR, next_u→csr_page. This instructionis used to load the base address of the memory region used to store thein-memory configuration state registers for a particular context (e.g.,processor, thread, etc.) in a register, such as TCBR. This address isthen used in processing that employs a base address, as describedherein. In this instruction, next_u→csr_page refers to the user datastructure that stores data for the context issuing the instruction. Thisdata includes the base address of the memory unit (e.g., page) storingthe in-memory configuration state registers. Although mtspr isspecified, other move to configuration state register (mtcsr)instructions may be used. Also, TCBR is just one example of a baseregister. Other base registers may be specified. Many variations arepossible.

In addition to the above instructions, two synchronization instructionsmay be provided to synchronize the cache with memory or in-processorregisters. For example, sync_o_csr is used to synchronize the cache andone or more in-memory configuration state registers; and sync_i_csr isused to synchronize the cache and one or more in-processor configurationstate registers.

As described herein, in one aspect, multiple configuration stateregisters are loaded or stored. There are no intervening exceptions;i.e., exceptions are avoided; the operation is either completed on allthe configuration state registers of the operation or none. Further,there are no page faults (e.g., pages are pinned; also once a page hasbeen loaded, references are guaranteed to the same page). If desired,hardware can make a process restartable; e.g., a load or store sequencein microcode or a state machine.

By using instructions to store/load multiple configuration stateregisters, certain expensive operations, such as ensure in-order pointand complete instruction in-order, are used less frequently, asdescribed with reference to FIGS. 17A-17C. The processing of FIGS.17A-17C is performed by a processor.

Referring to FIG. 17A, one example of using individual instructions(i.e., not a bulk operation) to store a plurality of configuration stateregisters to memory is described with reference to FIG. 17A. In oneexample, to move a configuration state register to memory, a move fromconfiguration state register instruction and a store instruction areused. For instance, in one example, the configuration state register isan SPR and the move from configuration state register instruction is anmfspr. Other configuration state registers and correspondinginstructions may be used.

In this example, based on execution of a mfspr instruction, such as aMove from SPR (mfspr) instruction, an in-order point is ensured, STEP1700. Then, contents of the configuration state register (e.g., SPR)specified by the instruction are copied from the configuration stateregister into, e.g., a general purpose register (GPR), STEP 1702. Theinstruction is completed in-order, STEP 1704. Thereafter, via a storeinstruction (STD), contents of the general purpose register are storedto memory, STEP 1706. This same process is repeated for eachconfiguration state register to be stored in memory (e.g., STEPS1708-1722), which may be many configuration state registers. Thus,requiring an ensure in-order point and complete instruction in-orderoperation for each configuration state register to be stored to memory.

However, in accordance with an aspect of the present invention, a storeconfiguration state register (ST_CSR) instruction is provided thatstores multiple configuration state registers in memory using a singleensure in-order point and a single complete instruction in-orderoperation, as described with reference to FIG. 17B.

Referring to FIG. 17B, based on execution of a store configuration stateregister instruction (ST_CSR), an in-order point is reached, STEP 1750.Then, contents of a selected configuration state register are loadedinto a temporary register, STEP 1752. Further, the contents of thetemporary register are then stored to memory (e.g., a memory controlpage), STEP 1754. The load/store operations are repeated one or moretimes 1756-1762 for one or more additional configuration stateregisters. Subsequent to copying the chosen configuration stateregisters to memory (which may be many such registers), the instructionis completed in-order, STEP 1770.

In one example, the ST_CSR instruction does not have an operand tospecify the registers to be copied; instead, all of the in-processorconfiguration state registers of the current context (e.g., process,thread, etc.) are copied. In another example, an operand may be includedand used to specify one or more configuration state registers to becopied to memory. Other variations are also possible.

In a further example, multiple configuration state registers may becopied from memory to in-processor using a bulk load operation (e.g.,LD_CSR).

Referring to FIG. 17C, based on execution of a load configuration stateregister (LD_CSR) instruction, an in-order point is ensured, STEP 1780.Then, contents of a selected configuration state register are obtainedfrom memory and loaded into a temporary register, STEP 1782. Then, thecontents of the temporary register are stored in a correspondingin-processor configuration state register, STEP 1784. The load/storeoperations are repeated one or more times 1786-1792 for one or more (andpossibly many) additional configuration state registers. Thereafter, theinstruction is completed in-order, STEP 1794.

In one example, the LD_CSR instruction does not have an operand tospecify the registers to be copied; instead, all of the configurationstate registers of the current context (e.g., process, thread, etc.) arecopied. In another example, an operand may be included and used tospecify one or more configuration state registers to be copied frommemory.

In one aspect, an instruction to perform a bulk operation performs thesame operation (e.g., a store, a load, etc.) on a group of configurationstate registers, in which the group is defined by a commoncharacteristic. The common characteristic may be, for instance, anumeric range of registers; having a same access permission or context(e.g., user, operating system, hypervisor); having a same functionalpurpose (e.g., exception handling, time keeping, etc.); or having a sameimplementation property (e.g., a set of configuration state registersbeing stored in-memory), as examples.

The use of bulk store and/or load operations improves processing withinthe computer. For instance, by using a bulk store operation toefficiently copy a plurality of configuration state registers to memory,context switch processing may be performed faster and more efficiently.Other benefits may also be realized.

In a further aspect, to facilitate processing, an architecturalplacement control is provided to indicate where in memory theconfiguration state registers are stored. For instance, hardware definesa set of controls to identify where in memory the configuration stateregisters are stored. As an example, at least one configuration stateregister is provided to specify the base address for storing theapplication state. As one example, the base address is a guest physicaladdress, i.e., the guest operating system specifies an address in itsown address space. For example, when an address is specified, aguest-level address (e.g., guest real, guest physical or guest virtualaddress) is specified, since allowing a guest to specify a host physicaladdress may compromise virtualization and security.

Further details regarding specifying an architectural configurationcontrol are described with reference to FIG. 18A. In one example, aprocessor performs this logic. Initially, an address indicative of amemory backing location (i.e., a base address) with respect to a presentexecution environment (e.g., application state, thread state, operatingsystem state, hypervisor state, particular guest or host operatingsystem level, etc.) is received, STEP 1800. For instance, a hypervisoruses hypervisor addresses (e.g., host virtual or absolute, physical,real addresses) and an operating system uses guest real or guest virtualaddresses with respect to the virtual machine/logical partition. Theprocessor obtains the address that indicates a location of the memorypage to store configuration state registers (i.e., the base address).

Optionally, that base address is translated to a physical address, STEP1802. (A translation may already have been performed or the address doesnot need to be translated.) In one embodiment, this translation maycause a page fault. In a further embodiment, collaborating hardware andsoftware are used to avoid the page fault, e.g., by pinning the pageprior to executing one or more aspects of the present technique.

Additionally, in one example, the translated address is captured, STEP1804. That is, in one example, the translated address is cached. Inanother embodiment, both the untranslated and translated addresses arestored.

In another embodiment with reference to FIG. 18B, a captured addresswith respect to the memory control (e.g., configuration state register)being referenced is obtained, STEP 1850. Additionally, the address maybe translated to a physical address, STEP 1852, and the in-memorycontrol (e.g., configuration state register) is accessed, STEP 1854. Thetranslation may cause a page fault, but using collaborating hardware andsoftware, the page fault may be avoided by, e.g., pinning the page priorto performing one or more aspects of the present technique.

In one embodiment, the base address is stored in a configuration stateregister, e.g., not in memory, to avoid a circular dependence. Otherexamples are possible.

Further, in one embodiment, the base address is translated to aphysical/real address; and in another embodiment, the base address istranslated to the next level's supervisory address (i.e., when anoperating system sets a page address, it is translated to a supervisoraddress). Other examples are possible.

As an example, both the untranslated and translated (physical/real) baseaddresses are stored. This eliminates the need to perform addresstranslation (e.g., dynamic address translation (DAT)) on everyconfiguration state register access and to handle page faults.

In one embodiment, the translated (real/physical) base address ismaintained in a processor register, and the untranslated base address ismaintained in an in-memory-configuration state register. In such anembodiment, the untranslated address is provided responsive to asoftware request to read out the base address of the configuration stateregister again. The translated address may be used to access such anaddress from its in-memory location. Other possibilities exist.

As described herein, a control, such as a configuration state register,is provided that includes a base address specifying where in memory oneor more in-memory configuration state registers are stored. Thesein-memory configuration state registers are registers architecturallydefined to be in-processor registers, but, in accordance with one ormore aspects of the present invention, have been converted to in-memoryconfiguration state registers. These in-memory configuration stateregisters are different than configuration values that are stored inmemory, since, at the very least, those values are not registers andthey are architecturally defined to be in-memory. They are notarchitecturally defined processor registers.

In a further aspect, efficiencies are achieved during a context switchbetween the program environment and, for instance, the operating systemor other supervisor environment and/or between different applications orthreads, etc. When a context switch is to be performed, data for aprevious context is saved. In one example, to save the context data,contents of the configuration state registers are read out of thein-processor registers and saved to memory. Then, the data for the nextcontext is loaded, which includes loading the configuration stateregisters to resume execution. This is an expensive process.

Even, in accordance with an aspect of the present invention, within-memory configuration state register execution, in which store queuebased speculation and out-of-order execution accelerate this process,there are still significant costs associated with saving and restoringcontext.

Therefore, in accordance with an aspect of the present invention, thecontext switch is performed by configuration state register pagemanipulation. In one example, the location of configuration stateregister in-memory storage is configurable. When switching contexts,instead of copying old configuration state register data out of aconfiguration state register memory unit (e.g., page) and copying newdata into the configuration state register page, a differentconfiguration state register memory unit (e.g., page) is selected,thereby changing the values of configuration state registers seen by theprocessor.

In accordance with an aspect of the present invention, a context switchis performed by modifying the base pointer or base address (referred toherein as base) included in, e.g., a base configuration state register(referred to herein as a base register) that indicates a location inmemory for one or configuration state registers (referred to herein asCSR backing memory), to avoid the need for unloading and reloading theconfiguration state registers.

There may be several types of context switches that may benefit fromthis aspect, including an operating system context switch, in which theoperating system performs a switch between different applications; ahypervisor context switch, in which a hypervisor or virtual machinemonitor switches between different partitions or virtual machines; and ahardware thread context switch between different hardware threads. Eachcontext switch affects different registers. For instance, when switchingout an application as part of an operating system used context switch, afew configuration state registers corresponding to the application arechanged to another location, but not other configuration state registers(e.g., not the operating system configuration state registers). Further,with hypervisor context switching, there may be more registers to beswitched out. Similarly, for a hardware thread context switch. Furtherdetails regarding a hardware thread context switch are described below.

In one embodiment, for a hardware thread context switch, the processoruses a thread scheduler to select from a plurality of threads loadedinto the processor. In accordance with an aspect of the presentinvention, however, hardware may select from a plurality of threads thatcan be scheduled by the hardware, in which the plurality exceeds thenumber of hardware thread contexts loaded in the processor. That is, inaccordance with an aspect of the present invention, the ability tocontext switch, as described herein, allows the hardware to use morethreads than loaded in the processor. The thread is selected, and thehardware schedules the thread by selecting that thread's in-memoryconfiguration information. In one embodiment, some of the executableregisters are stored in a register file on-chip or a fast second levelstorage. In another embodiment, general registers are also stored to anin-memory configuration page and loaded therefrom when a thread isscheduled. This is performed either on demand (e.g., each register whenfirst accessed), or in bulk (e.g., all registers at a scheduled time).

In one embodiment, rather than having a software agent (e.g., operatingsystem or hypervisor) decide to change a pointer to anotherconfiguration state register base, the hardware, responsive to ahardware criterion, adjusts the base pointer itself. One of theplurality of threads is selected in hardware and based on the pluralityof threads, one of the pointers to a system memory page havingconfiguration state registers is selected.

Further details relating to performing a context switch are describedwith reference to FIGS. 19A and 19B. FIG. 19A depicts one process forperforming a context switch, in which the data is copied; and FIG. 19Bdepicts another process for performing a context switch, in whichpointers are modified in accordance with an aspect of the presentinvention.

Referring initially to FIG. 19A, when a context switch is to beperformed, a context copy-out is initialized to copy-out the old contextdata, STEP 1900. This includes locating the context structure (e.g., asupervisor structure) for a previous context and identifying a firstconfiguration state register for the context to be read-out.

As examples, the context can be one of a virtual machine, a logicalpartition, a process, a thread, etc. The copy-out process continues.

The configuration state register to be stored is selected, STEP 1902. Inthis iteration, it is the configuration state register identified above.The selected configuration state register is read, STEP 1904, and thecontents of the configuration state register are stored to an in-memorystructure maintained by, e.g., the supervisor software (or hardware) forstoring context-switch data, STEP 1906. Next, a determination is made asto whether there are more configuration state registers to be contextswitched, INQUIRY 1908. If so, then processing continues with STEP 1902.

Subsequent to copying-out the data for the previous context, a copy-inprocess is performed for the new context. Thus, a context copy-in isinitialized, in which the context structure for the next context islocated and the first configuration state register for the contextwrite-in is identified, STEP 1910. The configuration state register tobe loaded in-memory is selected, STEP 1912, and the content for theselected configuration state register is read from a context structure(e.g., a supervisor structure), STEP 1914. The read context data iswritten to the configuration state register in-memory, STEP 1916. Adetermination is made as to whether there are more configuration stateregisters to be context switched, INQUIRY 1920. If so, then processingcontinues with STEP 1912. Otherwise, processing is complete.

Referring to FIG. 19B, another example of a context switch process inaccordance with an aspect of the present invention is described. In thisexample, instead of copying the data, pointers to the data aremanipulated. Initially, a context copy-out is initialized, in which thecontext structure (e.g., supervisor or hardware structure) for aprevious context is located, STEP 1950. Again, the context can be one ofa virtual machine, a logical partition, a process, a thread, etc. Next,in-processor configuration state registers are handled, STEP 1952, suchthat the contents of the registers are stored in memory. In oneembodiment, this is accomplished using, for instance, a copy loop, asdescribed with reference to FIG. 19A. In another embodiment, this isaccomplished using, for instance, a bulk copy operation (e.g., ST_CSR).

The address of the in-memory configuration state register data unit(i.e., e.g., the address of the memory page used to store theconfiguration state registers for this context (the base address)) maybe read, STEP 1954, and stored to a context structure, STEP 1956. In oneexample, this address does not change, so there is no need to repeatedlyread it and store it to the context structure. Instead, it is stored thefirst time or when that page is moved to a new location. The value isstored to an in-memory structure maintained, by, e.g., the supervisorsoftware or hardware, for storing context-switch data.

Subsequent to performing the copy-out process, a copy-in process isutilized to point to the new context data. Thus, a context copy-inprocess is initialized, in which the context structure (e.g., supervisoror hardware structure) is located for the next context and a firstconfiguration state register is identified for the context write-in,STEP 1960. The address for the configuration state register data unitfor the next context is loaded, STEP 1962. That is, for instance, theaddress of the memory page (the base address) to store the configurationstate registers for the new context is obtained. Additionally, thein-memory configuration state register data page address (base address)is written, STEP 1964. Further, the in-processor configuration stateregisters are handled, STEP 1966. As an example, the in-processorconfiguration state registers for this context are loaded from memory,e.g., using a copy loop or a bulk load (e.g., LD_CSR).

As described above, pointer manipulation may be used in contextswitching. This is also true for virtual machine migration.

In one embodiment, in-memory registers may be used to accelerate virtualmachine (or logical partition) migration and/or live machine migration.In one example, pages are migrated in accordance with conventionaltechniques; however, in-memory configuration state registers are notmoved. As an example, an ST_CSR instruction or operation is used tocapture the in-processor configuration state registers, but noconfiguration state registers are moved. Rather, the in-memoryconfiguration memory page is moved.

In a further embodiment, for live machine migration, the in-memoryconfiguration state register state(s) are moved, when the machine hasbeen quiesced. This may include multiple pages if there are multiplecontexts (e.g., multiple threads/processes etc.).

In one embodiment, when the host and target migration formats (e.g.,when configuration state registers are mapped to different offsetswithin an in-memory configuration register page by differentimplementation of an architecture) are not compatible, an adjustment isperformed by the migration agent. In one such embodiment, anarchitectural or micro-architectural version number of configurationstate register formats is provided, and the receiving system isresponsible for adjusting the layout. In another embodiment, a receivingand sending system negotiate a transfer format. In another embodiment, atransfer format is defined, e.g., a linear list of configuration stateregister values or a <key, value> pair, in which the key is aconfiguration state register number and the value is the value of theconfiguration state register.

In yet other embodiments, processors may support multiple versions oflayouts, adapting the remap logic based on an externally configuredconfiguration state register layout map (e.g., software specifies byloading a layout identifier into a configuration state register).

In at least some processor embodiments, the design may cache some valuesin custom in-processor locations for registers that are identified asin-memory registers. Thus, when performing a context switch, cachedcopies are to be synchronized, i.e., invalidate old cached values. Inone example, a context sync instruction (csync) is provided, whichindicates to the processor to invalidate old cached values. This isexecuted, for instance, each time a context switch is performed. In oneexample, the cached values corresponding to all the configuration stateregisters for a context are invalidated. In other examples, cachedvalues for specific registers are invalidated.

As described herein, on a context switch, a new page may be indicated,instead of copying the configuration state registers. This way, theconfiguration state register context image is already saved, and reducesthe context switch, at least for configuration state registersmaintained in-memory.

Further, in one embodiment, a context switch with page replace isenabled. A memory image (at least for those registers that should livein memory) is synced before loading a new memory context page.

In accordance with one embodiment, configuration state register syncinstructions are provided. Specifically, this may flush any cachedvalues, and suppress caching after the switch has been performed, andconversely may indicate that it may save values to the cache again.Example instructions include:

Sync outgoing CSRs: sync_o_CSR

Load new backing page mtspr TCBR, next_u→spr_page

Sync incoming SPRs sync_i_CSR

In another embodiment, a move to configuration state registerinstruction (e.g., mtspr instruction) to a base register (e.g., TCBR)automatically performs synchronization of cached outgoing and incomingvalues.

In a further aspect, based on loading a guest base address, whichindicates a location in memory used to store one or more in-memoryconfiguration state registers and may be stored in a base configurationstate register, address translation from the guest base address to acorresponding host base address is performed, in order to avoid thepotential of a page fault. This translation is performed, e.g.,immediately; i.e., based on receiving the base address and prior tousing the base address, e.g., during a storage reference. For example,when a virtual address is loaded into a base register (e.g., a load tobase, such as loading a guest base address in a configuration state baseregister (e.g., TCBR) to be used as a base address), the systemautomatically performs an address translation to a physical memoryaddress, and the translated physical address is captured in conjunctionwith the virtual address. In accordance with one architecturalembodiment, a load to the base register is identified as an instructioncausing or may cause a translation to be performed.

When an address cannot be translated, a translation fault is taken. Thepage is to be accessible for read and write in accordance with specifiedpermissions in, for instance, a page table entry (described below), or apage fault is raised.

Further details regarding automatically performing an addresstranslation based on execution of an operation that causes or may causea translation of a base address to be performed are described withreference to FIG. 20. This processing is performed by a processor, andin one example, this processing is performed based on execution of amove to configuration state register instruction (e.g., an mtsprinstruction) to a configuration state register memory base register. Itmay also be performed based on execution of other instructions.

Based on executing the instruction, an address indicative of a memorybacking location (e.g., a base address) with the respect to the presentexecution environment is received, STEP 2000. In accordance with onedefinition of the mtspr, the mtspr is defined to possibly incur adynamic address translation (DAT) page fault. Thus, in accordance withan aspect of the present invention, address translation is automaticallyperformed as part of the mtspr, even before determining a translation isneeded.

The received base address is translated to a physical base addressusing, e.g., DAT tables, an example of which is further described below,STEP 2002. A determination is made as to whether a DAT translation faulthas occurred, INQUIRY 2004. If a DAT translation fault has occurred, aDAT page fault is indicated, STEP 2006. A page fault handler softwareroutine is entered, STEP 2008, and the page fault handler is performed,STEP 2010. In one example, in accordance with a page fault handler, theinstruction is restarted, if it was a permissible fault (e.g., pagedout). Otherwise, the context or execution environment (e.g., operatingsystem, hypervisor, virtual machine, thread, process, etc.) receives anerror indication, possibly causing the context's termination.

Returning to INQUIRY 2004, if there is no DAT translation fault, theuntranslated base address is captured (e.g., stored in a register), STEP2012. Further, the translated base address is captured (e.g., cached),STEP 2014. Optionally, the subject page is pinned, STEP 2016.

Further details regarding one example of dynamic address translation aredescribed with reference to FIGS. 21A-21B. This processing is performedby a processor.

Dynamic address translation is the process of translating a virtualaddress into the corresponding real (or absolute) address. Dynamicaddress translation may be specified for instruction and data addressesgenerated by the CPU. The virtual address may be a primary virtualaddress, a secondary virtual address, an AR (Access Register)-specifiedvirtual address, or a home virtual address. The addresses are translatedby means of the primary, the secondary, an AR-specified, or the homeaddress space control element (ASCE), respectively. After selection ofthe appropriate address space control element, the translation processis the same for all of the four types of virtual addresses. An addressspace control element may be a segment table designation or a regiontable designation. A segment table designation or region tabledesignation causes translation to be performed by means of tablesestablished by the operating system in real or absolute storage.

In the process of translation when using a segment table designation ora region table designation, three types of units of information arerecognized—regions, segments, and pages. The virtual address,accordingly, is divided into four fields. In one example, for a 64-bitaddress, bits 0-32 are called the region index (RX), bits 33-43 arecalled the segment index (SX), bits 44-51 are called the page index(PX), and bits 52-63 are called the byte index (BX). The RX part of avirtual address is itself divided into three fields. Bits 0-10 arecalled the region first index (RFX), bits 11-21 are called the regionsecond index (RSX), and bits 22-32 are called the region third index(RTX), in one embodiment.

One example of translating a virtual address to a real address isdescribed with reference to FIG. 21A. This process is referred to hereinas a DAT walk (or a page walk) in which the address translation tablesare walked to translate one address (e.g., a virtual address) to anotheraddress (e.g., a real address). In this example, an address spacecontrol element (ASCE) 2100 includes a table origin 2102, as well as adesignation type (DT) control 2104, which is an indication of a startlevel for translation (i.e., an indication at which level in thehierarchy address translation is to begin). Using table origin 2102 andDT 2104, the origin of a particular table is located. Then, based on thetable, bits of the virtual address are used to index into the specifictable to obtain the origin of the next level table. For instance, if theregion first table (RFT) 2106 is selected, then bits 0-10 (RFX) 2108 ofthe virtual address are used to index into the region first table toobtain an origin of a region second table 2110. Then, bits 11-21 (RSX)2112 of the virtual address are used to index into region second table(RST) 2110 to obtain an origin of a region third table 2114. Similarly,bits 22-32 (RTX) 2116 of the virtual address are used to index intoregion third table (RTT) 2114 to obtain an origin of a segment table2118. Then, bits 33-43 (SX) 2120 of the virtual address are used toindex into segment table 2118 to obtain an origin of page table 2122,and bits 44-51 (PX) 2124 of the virtual address are used to index intopage table 2122 to obtain a page table entry (PTE) 2125 having a pageframe real address (PFRA) 2126. The page frame real address is thencombined (e.g., concatenated) with offset 2128 (bits 52-63) to obtain areal address. Prefixing may then be applied, in one embodiment, toobtain the corresponding absolute address.

Another example of address translation is described with reference toFIG. 21B. In this example, a DAT walk is performed to translate aninitial guest virtual address to a final host real address. In thisexample, address space control element (ASCE) 2100 is a guest addressspace control element, and DT 2104 of ASCE 2100 indicates that guesttranslation determined by guest address translation structure 2160 is tostart at region first table 2106 pointed to by table origin 2102. Thus,the appropriate bits of the initial guest virtual address (e.g., RFX2108) are used to index into region first table 2106 to obtain a pointerof an entry of the region first table. The address of the region firsttable entry (RFTE) is a guest real or absolute address. This guest realor absolute address, with the main storage origin and limit applied,when appropriate, corresponds to a host virtual address. Thisintermediate host virtual address is then translated using host addresstranslation structures 2170. In particular, address space controlelement (ASCE) 2150 is a host address space control element used toindicate a start level for translation in host address translationstructures 2172. Based on the start level (e.g., region first table)indicated by DT 2154, the particular bits of the host virtual addressare used to index into the indicated table with table origin 2152 to beused for translation using host address translation 2172, as describedwith reference to FIG. 21A. The translation of the host virtual addresscorresponding to the guest RFTE continues until a host page frame realaddress (PFRA) 2174 a is obtained.

Data at the intermediate host page frame real address is a pointer tothe next level of guest address translation structures (e.g., guestregion second table 2110, in this particular example), and translationcontinues, as described above. Specifically, host address translationstructures 2176, 2178, 2180 and 2182 are used to translate theintermediate host virtual addresses associated with the guest regionsecond table 2110, region third table 2114, segment table 2118 and pagetable 2122, respectively, resulting in host PFRAs 2174 b, 2174 c, 2174 dand 2174 e, respectively. Host page frame real address 2174 e includesthe address of a guest page table entry 2125. Guest page table entry2125 includes a guest page frame real address 2126, which isconcatenated with the offset from the initial guest virtual address toobtain the corresponding guest absolute address. In some cases, the mainstorage origin and limit are then applied to calculate the correspondinghost virtual address, which is then translated, as described above,using address translation structures 2184 to obtain host page frame realaddress 2174 f. The host page frame real address is then combined (e.g.,concatenated) with the offset (e.g., bits 52-63) of the host virtualaddress to obtain the final host real address. This completestranslation of a guest virtual address to a host real address.

Although in the above examples, translation starts at the region firsttable, this is only one example. Translation may start at any level foreither the guest or the host.

In one embodiment, to improve address translation, the virtual addressto real or absolute address translation mapping is stored in an entry ofa translation look-aside buffer (TLB). The TLB is a cache used by thememory management hardware to improve virtual address translation speed.The next time translation for a virtual address is requested, the TLBwill be checked and if it is in the TLB, there is a TLB hit and the realor absolute address is retrieved therefrom. Otherwise, a page walk isperformed, as described above.

As indicated, guest translations may be included in the TLB. Theseentries may be composite guest/host entries which implicitly include oneor more host translations. For example, a guest virtual TLB entry maybuffer the entire translation from the initial guest virtual addressdown to the final host real or absolute address. In this case, the guestTLB entry implicitly includes all intermediate host translations 2172,2176, 2178, 2180 and 2182, as well as the final host translation 2184,as described in FIG. 21B above. In another example, a hierarchical TLBmay contain an entry in a first level of the TLB which buffers atranslation from the initial guest virtual address down to theassociated origin of the guest page table 2122 and a separate entry froma second level of the TLB which buffers the translation from the guestpage table entry address down to the final host real or absoluteaddress. In this example, guest entries in the first level of the TLBimplicitly include intermediate host translations 2172, 2176, 2178 and2180 which correspond to the host translations which back guest regionand segment tables, and guest entries in the second level implicitlyinclude intermediate host translation 2182 which backs the guest pagetable and final host translation 2184, as described in FIG. 21B. Manyimplementations of a translation look-aside buffer are possible.

In the above examples, the page frame real address is included in a pagetable entry of a page table. The page table includes one or moreentries, and further details of a page table entry are described withreference to FIG. 22.

In one example, a page table entry (PTE) 2200 is associated with aparticular page of memory and includes, for instance:

-   -   (a) Page Frame Real Address (PFRA) 2202: This field provides the        leftmost bits of a real (e.g., host real) storage address. When        these bits are concatenated with the byte index field of the        virtual address on the right, the real address is obtained.    -   (b) Page Invalid Indicator (I) 2204: This field controls whether        the page associated with the page table entry is available. When        the indicator is zero, address translation proceeds by using the        page table entry. When the indicator is one, the page table        entry cannot be used for translation.    -   (c) Page Protection Indicator 2206: This field controls whether        store accesses are permitted into the page.    -   (d) Pinning indicator 2208: This field is used, in accordance        with an aspect of the present invention, to indicate whether        this page is to be pinned. In one example, a one indicates it is        to be pinned, and a zero indicates it is not to be pinned.

A page table entry may include more, fewer and/or different fields thandescribed herein. For instance, in the Power Architecture, the PTE mayinclude a reference indicator that indicates whether a correspondingblock of memory has been referenced, and/or a change indicator thatindicates that a corresponding block of memory has been stored into.Other variations are possible.

In yet a further aspect of the present invention, configuration stateregisters are separated and assigned based on host and guest attributes,context and/or execution environment (e.g., thread state, applicationstate, operating system state, hypervisor state, particular guest orhost operating system level, etc.), as examples, in order to enableincreased management flexibility. As examples, configuration stateregisters may be separated by hypervisor, operating system, application,thread number or other execution environments, etc.

As particular examples, hypervisor privilege configuration stateregisters are stored in a hypervisor assigned unit of memory (e.g.,page); operating system privilege configuration state registers arestored in an operating system unit of memory (e.g., page), and so forth.Further, when multiple threads are supported and configuration stateregisters are replicated for each thread, then separate units of memory(e.g., pages) may be supported for each thread. An example of suchseparation is depicted in FIG. 23.

As shown in FIG. 23, one set of configuration state registers 2300 isused by a thread or process 2302; another set of configuration stateregisters 2304 is used by an operating system 2306; a yet further set ofconfiguration state registers 2308 is used by a hypervisor 2310; andstill another set of configuration state registers 2312 is used by ahypervisor 2314. Other examples are also possible.

In one example, the configuration registers for a particular executionenvironment are statically defined for that execution environment, andinclude those registers readable or writable by the executionenvironment. In a further example, the registers are dynamicallyassigned based on use. Other examples are also possible.

In one embodiment, a separate memory region (e.g., as a multiple of anallocatable translation unit) is assigned to each separatelycontrollable execution environment (e.g., thread, process, operatingsystem, hypervisor), and therefore, the set of configuration stateregisters associated with that execution environment is assigned to thatmemory region. As an example, configuration state registers are assignedto a corresponding memory area, based on logical ownership, since someconfiguration state registers may be accessible from multiple executionenvironments (e.g., read accessible from an operating system, andread/write (R/W) from a hypervisor).

Although different execution environments may have different privilegelevels, in one aspect, a higher level privilege level has access controlto lower levels. A specific privilege level may be specified with LD_CSRand ST_CSR instructions, and sync operations described herein.

In one example, the configuration state register numbers are remapped,as described above; i.e., indices of configuration state registernumbers are compacted to co-locate registers with respect to eachgrouping.

By providing a specific set of registers per execution environment,certain types of processing are facilitated, including contextswitching. As described above, the assigning of specific sets ofregisters based on execution environment and assigning separate memoryunits to those sets, facilitates the managing of the registers, as wellas processes that use those registers, including context switching.

As described herein, a context switch may be performed by changing abase address pointer in a configuration state base register, rather thanunloading and reloading the configuration state register state. Toaccomplish this, a user mode state is to be independently switchablefrom supervisor state in order to switch user contexts; an operatingsystem state is to be independently switchable from hypervisor state inorder to switch virtual machines; and a per-hardware thread orsubprocessor state is to be independently switchable, if those are to beswitched independently from other threads/sub-processors. The separatememory regions and separately assignable configuration state registersfacilitate this.

In accordance with one aspect of the present invention, separateconfiguration state base registers designate the location (base) of eachgrouping of in-memory configuration state registers. Further, in atleast one embodiment, access control to each of the base registers is tohave suitable access permissions. As examples, for contexts to beswitched by the operating system, an operating system privilege is aminimum prerequisite to modify the base register; and for contexts to beswitched by the hypervisor, a hypervisor privilege is the minimumprerequisite to modify such base registers, and so forth.

In a further aspect, a capability is provided to prevent the moving of amemory unit (e.g., a page) by host level software (such as a hypervisoror virtual machine monitor) that provides the storage for one or moreconfiguration state registers (i.e., the CSR backing memory). In oneexample, this includes pinning the memory unit and providing anindication of automatic pinning for the CSR backing memory.

When a configuration state register indicating the base of memory thatprovides the storage for one or more configuration state registers(i.e., the base register, such as TCBR) is written, an indication isprovided to the host of the present guest, in accordance with anarchitectural specification. In at least one embodiment, the indicationcorresponds to an exception. In one embodiment, an exception typeindicates a write event to a configuration state base register.Responsive to receiving an indication of a configuration state baseregister change, at least one host supervisor software (e.g., ahypervisor or a virtual machine monitor) performs operations to updatepage pinning information. In one aspect of the invention, updatingpinning information includes recording the address of a pinned CSRbacking memory page or setting a pinning indicator corresponding to thepage. In another aspect, updating pinning information further includesunpinning of a previously pinned CSR backing memory page by removing apreviously recorded address for a particular configuration state baseregister from a pool of one or more recorded addresses corresponding topinned CSR backing memory or resetting the pinning indicator.

In accordance with one aspect of the present invention, these updatesensure that the one or more host levels do not page out or move CSRbacking memory, thereby invalidating a cached address translation, orotherwise causing an update to a configuration state register causing apage translation fault. In another aspect of the present invention,pinning information may also be used to move the location of CSR backingmemory, by providing notice of its location, and giving one or morehosts the opportunity to update any cached translations.

Further, in order to support multi-level guest schemes, the instructionthat initializes the base register that points to the memory backing theconfiguration state registers (the CSR backing memory) may further bespecified to transitively raise exceptions at multiple levels of host soas to ensure suitable pinning. In one embodiment, only one level of hostis notified, and that host will cause pinning by HCALLs (hypervisorcalls) as/when appropriate.

By pinning memory backing pages for configuration state registers inmemory, page faults are avoided when a configuration state register isaccessed. This may be unexpected by the software, and cause panic( ) insome software, such as for example, when software checks whatinstruction caused a trap and finds it was an instruction not defined toaccess memory that caused a data page fault. Panic( ) is a call inoperating systems performed when an unexpected event happens and usuallyresults in a system crash.

Pinning is also used, for instance, to avoid circular exceptions (e.g.,when the page used for exception-related configuration state registersis not available, a page fault exception for that page would have to beraised, etc.); and to ensure a fast response (e.g., to exceptions andother external events that involve configuration state registerhandling).

In one embodiment, pinning is performed in software. For instance, asdescribed below, pinning may be performed using hypervisor calls (HCALL)in conjunction with a software context switch in a paravirtualizedenvironment.

In one embodiment, when a context (e.g., a thread context, or a processcontext, or a logical partition context, or a virtual machine context,or an operating system context, and so forth) is initialized bysupervisory software, the supervisor allocates memory to provide storagefor one or more configuration state registers corresponding to thecontext being initialized.

In one example, this may be performed by calling an allocation routineproviding memory of suitable alignment and size. In accordance with atleast one embodiment, the returned address is stored in a memory areastoring information corresponding to the context. In one embodiment,this memory area is referred to as the “u area” and denoted by thevariable “u”. In at least one embodiment, the variable u is a record,struct, class or other composite data type with a plurality of memberscorresponding to various attributes to be recorded for a context. In atleast one embodiment, this structure includes a member (field)corresponding to the address of at least one CSR backing memory page. Inat least one example, this member is named “csr_page”.

my_csr_page_pointer=allocate backing page

u.csr_page=my_csr_page_pointer

When performing pinning of CSR backing memory using e.g., HCALLs to ahypervisor in a paravirtualized environment, the context switch contextsequence (e.g., a sequence in accordance with one of FIGS. 19A and 19B)is augmented with pinning and unpinning HCALLs. In accordance with oneembodiment of a context switch with this aspect of the presentinvention, the following steps are performed:

(1) Save non-CSR state of previous context, including but not limitedto, general purpose registers, floating point registers, vectorregisters, etc., in accordance with known techniques;

(2) Save in-processor configuration state registers (e.g., based on thetechniques of one of FIGS. 19A and 19B);

(3) Pin incoming CSR backing memory page (i.e., page being activated asconfiguration state register memory page as part of switching in(activating) the next context): HCALL(PIN, next_u→csr_page), in whichnext_u is a pointer pointing to the u area of the context being switchedin (activated) as next context;

(4) Optionally, in at least one embodiment, sync outgoing configurationstate registers; sync_o_csr;

(5) Load base register with the base address of the CSR backing memorycorresponding to the context being activated (in one example, this CSRcorresponds to TCBR): mtspr TCBR, next_u→csr_page;

(6) Optionally, in at least one embodiment, sync incoming configurationstate registers: sync_i_csr;

(7) Unpin outgoing CSR backing memory page (i.e., page being deactivatedas CSR backing memory page as part of switching out (deactivating) theprevious context): HCALL(UNPIN, prev_u→csr_page), in which prev_u is apointer pointing to the u area of the context being switched out(deactivated) as previous context;

(8) Load other non-CSR state of next context, including but not limitedto, general purpose registers, floating point registers, vectorregisters, etc., in accordance with known techniques;

(9) Transfer control to the newly activated context, e.g., transfer fromoperating system to application thread or process context using, e.g.,the rfid instruction (in an implementation in conjunction with PowerISA).

Those skilled in the art will understand that the steps hereinabove maybe reordered. For example, the unpin operation may be performed prior tothe pinning operation, in at least one embodiment. Other variations arepossible.

In another embodiment, pinning is performed responsive to loading a baseregister, such as TCBR. A notification event (e.g., interrupt) may beraised to a supervisor. If there are multiple supervisor levels, anotification event is raised to each supervisor level, in oneembodiment. Responsive to receiving a notification event, pinninginformation is updated.

In another embodiment, writing a value to a base register causes a pagetable entry indication flag indicating a pinned page to be set in acorresponding PTE. If there are multiple levels of page table entry, apage table entry indication flag may be set for each level.

In at least another embodiment, at least one of a pin and an unpininstruction is provided that initiates a pin or unpin process.

In yet another embodiment, pinning may be determined by host software byinspecting base registers active in the system to determine which pagesare “pinned”, i.e., the plurality of contents of base registers (e.g.,TCBR) represents the record of pinned pages. In at least one embodiment,before moving or paging out a page, supervisor level software determineswhether a page corresponds to a pinned page by determining whether apage address corresponds to an address in at least one of the baseregisters.

In yet a further embodiment, a host may receive a notification event ofpinning, e.g., as an exception in one example embodiment. Based onreceiving the notification, the host system receives an address to bepinned and stores it for future reference during memory management. Inone embodiment, a notification also includes information about aprevious address that is being unpinned (e.g., the previous value storedin the base configuration state register, or another value otherwisesupplied, e.g., using a configuration state register).

Further details regarding one example of providing a pinningnotification to the host are described with reference to FIG. 24. Thislogic is performed by a processor. Referring to FIG. 24, a newconfiguration value (e.g., a guest address) is received for a memorypage (or other unit of memory) containing in-memory configuration stateregisters, STEP 2400. In one example, this new configuration value isstored in a base register (such as, e.g., TCBR). However, in otherembodiments, the value may be provided in another manner.

The guest address (e.g., a guest virtual address or a guest realaddress) of the memory page is translated, STEP 2402. In one example,the guest address is translated to a physical real address and thetranslation is cached for future access. A variable n is set equal tothe guest level, STEP 2404. Then, n is decremented by a select value,e.g., 1, STEP 2406. Host level n is notified of a pinning event usingthe host level n virtual address corresponding to the guest addressbeing pinned, STEP 2408. Further, a determination is made as to whetherthere are more host levels (e.g., is n greater than 0), INQUIRY 2410. Ifthere are more host levels, then processing continues with STEP 2406.Otherwise, processing is complete.

In one embodiment, the pinning is indicated by an indicator, such as abit, in a page table entry corresponding to the address. The bitindicates that the page is pinned and in use by a guest. One example ofthis pin indicator is depicted in FIG. 22.

Further details relating to pinning are described with reference to theexample translate and pin operations depicted in FIG. 25. In oneexample, a processor performs this processing. Initially, a newconfiguration value (e.g., guest address) for a memory page containingin-memory configuration state registers is received, STEP 2500. Theguest address is translated to a physical address and the translation iscached for future access, STEP 2502. A variable n is set equal to theguest level, and ADDRESS is set equal to the guest virtual address, STEP2504. Thereafter, n is decremented by a defined value, such as 1, andADDRESS is set equal to translate_to_host (ADDRESS, n), STEP 2506. Thatis, ADDRESS is set to the translated host address for the host level.The pinning indicator (e.g., bit) is set (e.g., set to one) in the pagetable entry for the address, STEP 2508. Further, a determination is madeas to whether there are more host levels; that is, is n greater thanzero, INQUIRY 2510? If there are more host levels, then processingcontinues with STEP 2506. Otherwise, processing ends. At this point,ADDRESS corresponds to the physical address of a pinned page and can beused for synergy with address translation.

Based on translation and caching, the indicator (e.g., bit) is set, inone example, at all host levels. In one embodiment, page table walkingand pin indication are combined. This improves performance sincetranslation accesses the same page table entries used for pinindication.

In one embodiment, unpinning is performed on another value (e.g., theprevious value (address) stored in the configuration state register, oranother value otherwise supplied, e.g., using a configuration register).

One example of processing relating to translate and unpin operations isdescribed with reference to FIG. 26. In one example, a processorperforms this processing. Initially, a request is received to unpin anaddress, STEP 2600. This request includes a guest virtual address to beunpinned. Further, n is set equal to the guest level, and ADDRESS is setequal to the guest virtual address to be unpinned, STEP 2602. Next, n isdecremented by a defined value, such as 1, and ADDRESS is set equal totranslate_to_host (ADDRESS, n), STEP 2604. That is, ADDRESS is set tothe translated host address for the host level. The pinning indicator(e.g., bit) in the page table entry for the address is reset (e.g., setto zero), STEP 2606. Thereafter, a determination is made as to whetherthere are more host levels (e.g., is n greater than 0), STEP 2608. Ifthere are more host levels, then processing continues with STEP 2604.Otherwise, processing ends. At this point, ADDRESS corresponds to thephysical address of the unpinned page.

As described herein, based on determining that a unit of memory is to bepinned, notification is, e.g., automatically provided. The notificationmay be by setting an indicator, raising an interrupt, providing anexception, etc. Many variations are possible.

In a further aspect, efficient pinning management is provided viaparavirtualized pinning calls. It is desirable to not have to pin andunpin pages every time they are installed. On the other hand, it is alsodesirable to limit the number of pinned pages so as to not unnecessarilyfragment the host's page cache, and limit its page allocation freedom.Consequently, pinning HCALLs (hypervisor calls) are introduced in whicha guest specifies a page to be unpinned by a host. A hypervisor canindicate whether a page to be unpinned was unpinned or not, therebygiving the guest the flexibility to not have to call a pin request forevery page, if the hypervisor has resources available.

This call includes, in one embodiment, updating the base pointer oraddress (base) to the CSR memory backing page. Further, in oneembodiment, the guest specifies whether it would like to retain a pageas pinned.

In another embodiment, a hypervisor may request, by callback to anoperating system, the return of pinned pages that have been previouslyleft to the operating system when the hypervisor runs into a lowresource situation. In one embodiment, the operating system specifiesthe one or more pages it would like to unpin as a response to thecallback.

In accordance with one or more aspects, a single call, such as oneHCALL, is used to perform pin and unpin operations by, e.g., a hostexecuting on a processor, as described with reference to FIG. 27. Asdepicted, in one example, a pin operation 2700 and an unpin operation2702 are performed in response to one hypervisor call 2704. In oneexample, an unpin operation is performed on a first address (e.g., afirst base address) to unpin an old page, STEP 2710, and a pin operationis performed on a second address (e.g., a second base address) to pin anew page, STEP 2720. The one call is used, instead of multiple calls,saving processing time. The number of unpin and pin calls are reduced byhaving a combined pin and unpin call, specifying a new page (e.g., CSRbacking memory page) to be pinned, and specifying a previous page to beunpinned.

In one embodiment, the operating system can request that the addressspecified in the call to be unpinned not be unpinned, if hypervisorconsole management constraints allow this. A response on whether theaddress is pinned or unpinned is returned. Later, the hypervisor canstill use callback to request the operating system to have one or morepinned pages unpinned.

In one example, the operating system holds more than the number of pagesnecessary for active in-memory configuration state register pages. In afurther example, the operating system pins all pages holding in-memoryconfiguration state registers, whether active or not. This eliminatesthe need for future pinning. However, this may lead to an excessivenumber of pinned pages and system inefficiency. Therefore, in oneembodiment, the operating system offers a callback function, in whichthe hypervisor can call the operating system to deallocate pinned pageswhen too many pages (or a number greater than a selected number) arepinned in a system for in-memory configuration state register use.

Further details relating to unpinning/pinning are described withreference to the below example in which unpinning/pinning are performedin a context switch. In particular, the below example describes oneexample of context pinning in which a guest (OS) requests a host (HV) toswitch pinning, and further optionally, retain a pin.

In one embodiment, when a context (e.g., a thread context, or a processcontext, or a logical partition context, or a virtual machine context,or an operating system context, and so forth) is initialized bysupervisory software, the supervisor allocates memory to provide storagefor one or more configuration state registers corresponding to thecontext being initialized.

In one example, this may be performed by calling an allocation routineproviding memory of suitable alignment and size. In accordance with atleast one embodiment, the returned address is stored in a memory areastoring information corresponding to the context. In one embodiment,this memory area is referred to as the “u area” and denoted by thevariable “u”. In at least one embodiment, the variable u is a record,struct, class or other composite data type with a plurality of memberscorresponding to various attributes to be recorded for a context. In atleast one embodiment, this structure includes a member (field)corresponding to the address of at least one CSR backing memory page. Inat least one example, this member is named “csr_page”.

my_csr_page_pointer=allocate backing page

u.csr_page=my_csr_page_pointer

When performing pinning of CSR backing memory using e.g., HCALLs to ahypervisor in a paravirtualized environment, the context switch contextsequence (e.g., a sequence in accordance with one of FIGS. 19A and 19B)is augmented with pinning and unpinning HCALLs. In accordance with oneembodiment of a context switch with this aspect of the presentinvention, the following steps are performed:

-   -   (1) Save non-CSR state of previous context, including but not        limited to general purpose registers, floating point registers,        vector registers, etc., in accordance with known techniques;    -   (2) Save in-processor configuration state registers (e.g., based        on the techniques of one of FIGS. 19A and 19B);    -   (3) Optionally, in at least one embodiment, sync outgoing        configuration state registers; sync_o_csr;    -   (4) If not (next_u→csr_page_pinned)        -   Find page to unpin victim=select_TCBR_for_unpin( )        -   Desirable to retain? retain=retain_desirable_p(victim);        -   Give to get pin page lost_victim=HCALL(PIN_give_to_get,            next_u→csr_page, victim, retain)        -   Victim was lost? if (lost_victim) mark unpinned (victim);    -   (5) Optionally, in at least one embodiment, sync incoming        configuration state registers: sync_i_csr;    -   (6) Load other non-CSR state of next context, including but not        limited to general purpose registers, floating point registers,        vector registers, etc., in accordance with known techniques;    -   (7) Transfer control to the newly activated context, e.g.,        transfer from operating system to application thread or process        context using, e.g., the rfid instruction (in an implementation        in conjunction with Power ISA).

Further details of one example of performing pin/unpin operations aredescribed with reference to FIG. 28. A request to pin a new address, NA,and to unpin an old address, OA, is received via, e.g., a singlehypervisor call, as well as a retain indicator (or an indication ofsuch) specifying whether a request is being made to retain OA as pinnedmemory, STEP 2800. As an example, the new address indicates a CSR memorybacking page. Address NA is pinned, STEP 2810, and a determination ismade as to whether the retain indicator specifies a request to retainaddress OA as pinned in memory, INQUIRY 2820. If there is a request toretain the pinning of OA, a determination is made as to whether to grantthe maintaining of the pinned page based on an indicated policy (e.g.,resource allocation across multiple virtual machines) and availablememory for pinning, INQUIRY 2822. If the request is granted, INQUIRY2824, an indication that an unpin operation is not to be performed ismade, STEP 2826. Otherwise, address OA is unpinned, STEP 2830, and anindication is provided that the unpin is performed, STEP 2832.

Returning to INQUIRY 2820, if the retain indicator specifies that thepinning is not to be retained, then processing continues with STEP 2830.

As described above, one call may be used to unpin one address, pinanother address and/or request that the address to be unpinned actuallynot be unpinned. By using one call, processing is facilitated andperformance is improved.

As described herein, selected configuration state registers are storedin-memory, instead of in-processor. By storing the registers in-memory,certain benefits and optimizations may be achieved, including thoseassociated with data corruption detection and correction.

In one example, memory-backed state is used to enhance resiliency and toaddress single event upsets (SEU) or soft errors. Single event upsetsare state changes introduced by the effect of ionizing radiation. AsCMOS (complementary metal-oxide-semiconductor) feature sizes shrink, theamount of charge QCRIT used to change a bit shrinks with it, as lesscharge is stored for each bit. To remediate the impact of single eventupsets, data protection is applied. This includes, for instance, usingparity or error correction code (ECC) protection of registers, to detectand repair, respectively, damaged state register values. Errorcorrection code or parity protection is used, as repair is reasonablyaffordable for register files when the design across an area can beamortized over many registers. For in-processor configuration registers,oftentimes this is unaffordable because separate protection and recoverywould have to be designed for each register.

However, in accordance with an aspect of the present invention,configuration state registers are stored in-memory where they areprotected with one or more advanced protection mechanisms, including,but not limited to, parity bits and error correction codes (ECC), inaccordance with an aspect of the present invention.

In one aspect, in-processor configuration state registers are alsoprotected by using the SEU-resilient system memory hierarchy. In oneembodiment, in-processor configuration state registers are protectedwith a technique to detect SEU-induced corruption. A variety ofdetection techniques may be used in conjunction with aspects of thepresent invention. In one example, the corruption detection mechanismcorresponds to the use of data parity protection for in-processorconfiguration state registers. In another embodiment, SEU-inducedcorruption may be detected by testing for signatures of register valuechanges in the absence of a write to a configuration register. Inaddition, such in-processor registers are also stored in-memory toensure an ECC-protected copy of the in-processor configuration registersis available. In one embodiment, responsive to updates of thein-processor configuration registers which are so protected, a copy ofthe update is also stored to the in-memory copy. When the processorrecognizes a parity error, the value is retrieved from the ECC protectedin-memory copy and the in-processor register is updated.

Further, in one embodiment, high use-rate values, such as instructionaddress, data address and content break point registers, are stored inbacking memory and can be recovered when a single event upset isdetected. Recovery includes parity protection, either via a hardwarereload path or by performing a machine check, and having a machine checkhandler reload those registers. Thus, in accordance with an aspect ofthe present invention, configuration register state is protected bystoring the configuration state registers in system memory and using,e.g., ECC, to protect the single event upsets.

Further details relating to using error correction code forconfiguration state registers are described below. In particular,examples of using error correction code for data writes are describedwith reference to FIGS. 29A-29C, and examples of using error correctioncode for data reads are described with reference to FIGS. 30A-30C.

Referring initially to FIG. 29A, in this example, a data write isunprotected. In this example, a value is received for an in-processorconfiguration state register, STEP 2900, and that value is written to alatch implementing the configuration state register, STEP 2902. Thisdata write is unprotected for single event upsets and other types oferrors.

In contrast, with reference to FIG. 29B, a value is received for anin-processor configuration state register, STEP 2920, and errorprotection or an error correction code is computed, STEP 2922. Thereceived value is written to the configuration state register (e.g., thelatch implementing the configuration state register) in conjunction withthe protection or error correction code, STEP 2924.

Yet further, with reference to FIG. 29C, one embodiment of a data writefor an in-memory configuration state register is described. In thisexample, a value for the in-memory configuration state register isreceived, STEP 2952, and a determination is made as to the system memoryaddress at which the configuration state register is stored, STEP 2954.The value is then stored to that system memory address which isprotected, since it is part of memory benefitting from error protection,STEP 2956.

In one aspect, the storing includes, for instance, computing an errorcorrection code for a received in-memory configuration state registervalue, and storing the computed error correction code with the receivedvalue. As known, an error correction code adds one or more parity bitsto the data bits representing the value (e.g., one or more parity bitsper one or more subsets of the data bits) and uses those parity bits todetermine any errors. If there is an error in the data, the parity bitsindicate where in the data bits there is an error, allowing a correctionto be made (e.g., change one or more data bits, represented in binary,to another value).

In addition to the above, one example of performing data reads forin-processor configuration state registers and in-memory configurationstate registers are described with reference to FIGS. 30A-30C.

Referring to FIG. 30A, one example of a data read for an in-processorconfiguration state register, in which no protection is offered isdescribed. In this example, the value is provided to the processorlogic, STEP 3000.

In contrast, one example of processing associated with reading a valuefrom an in-processor configuration state register in which protection isprovided is described with reference to FIG. 30B. In this example, thevalue is received from the latch, STEP 3020, and the correction code ischecked, STEP 3022. If corruption is not detected, INQUIRY 3024, thevalue is provided to the processor logic, STEP 3030. However, returningto INQUIRY 3024, if corruption is detected, then the corrected value iscomputed, STEP 3026, and the corrected value is written back to thelatch, STEP 3028. Further, that value is provided to the processorlogic, STEP 3030.

Additionally, a data read that uses an in-memory configuration stateregister is described with reference to FIG. 30C. In this example, aconfiguration state register number is received, STEP 3050. Adetermination is made as to the system memory address for theconfiguration state register, STEP 3052. The value is read from theprotected system memory, STEP 3054, and that value is provided to theprocessor logic, STEP 3056.

In one aspect, the reading includes determining, using the errorcorrection code, whether corruption of the data (e.g., the value) hasoccurred. If corruption is detected, one or more actions may be taken,including but not limited to, performing recovery. Recovery may includecomputing a corrected value for the corrupted value using the errorcorrection code. Other examples also exist.

By using in-memory configuration state registers, error protectionbenefits are received, avoiding expensive additional steps and latencyto add such protection to the latches implementing in-processorconfiguration state registers.

As examples, error correction is provided based on writing to anin-memory configuration state register. Error detection and correctioncode may be generated based on receiving a value of a configurationstate register to be stored in memory. Error detection and correctioncode may be used to detect whether corruption has occurred, based onreceiving a read request for memory storing a configuration stateregister. Corruption is corrected, based on reading an in-memoryconfiguration state register and detecting corruption has occurred.

Described in detail herein are aspects related to providing in-memoryconfiguration state registers. By providing configuration stateregisters in-memory, processing is facilitated, and performance may beenhanced. Certain improvements and optimizations may be realized.

The in-memory configuration state registers may be used in instructionprocessing, as well as in other operation sequences. For example, anexception may be received based on receiving an interrupt signal from anexternal device. In processing the exception, one or more configurationstate registers, such as a SRR0 and SRR1, may be accessed. When theseregisters are in-memory configuration state registers, the interruptprocessing sequence is expanded to include load and/or store operations.Other examples and/or variations are possible.

One or more aspects of the present invention are inextricably tied tocomputer technology and facilitate processing within a computer,improving performance thereof. Further details of one embodiment offacilitating processing within a computing environment, as it relates toone or more aspects of the present invention, are described withreference to FIGS. 31A-31B.

Referring to FIG. 31A, in one embodiment, a determination is made that aunit of memory (e.g., a page) that provides storage for one or morein-memory configuration state registers is to be pinned in a system thatincludes a guest operating system and one or more hosts at one or morehost levels (3100). Based on determining that the unit of memory is tobe pinned, notifying at least one host of the one or more hosts that theunit of memory is to be pinned (3102). In one example, the determiningis performed based on receiving a new configuration value for the unitof memory (3104). The new configuration value is, for instance, a guestaddress used to locate the unit of memory (3106).

In a further aspect, a host level of the one or more host levels isselected (3110), the guest address is translated to a host address forthe selected host level (3112), and the selected host level is notifiedof the pinning (3114). In yet a further aspect, the selecting, thetranslating and the notifying are repeated for each host level of theone or more host levels (3116).

As one example, with reference to FIG. 31B, the notifying includessetting an indicator in a page table entry corresponding to the selectedhost level (3118). As another example, the notifying includes setting anindicator in a page table entry corresponding to the unit of memory(3120). In yet a further example, the notifying includes raising aninterrupt (3122).

In a further aspect, a determination is made that the unit of memory isto be unpinned (3124), and based on determining that the unit of memoryis to be unpinned, an indicator used to indicate pinning is reset(3126).

Other variations and embodiments are possible.

Other types of computing environments may also incorporate and use oneor more aspects of the present invention, including, but not limited to,emulation environments, an example of which is described with referenceto FIG. 32A. In this example, a computing environment 20 includes, forinstance, a native central processing unit (CPU) 22, a memory 24, andone or more input/output devices and/or interfaces 26 coupled to oneanother via, for example, one or more buses 28 and/or other connections.As examples, computing environment 20 may include a PowerPC processor ora pSeries server offered by International Business Machines Corporation,Armonk, N.Y.; and/or other machines based on architectures offered byInternational Business Machines Corporation, Intel, or other companies.

Native central processing unit 22 includes one or more native registers30, such as one or more general purpose registers and/or one or morespecial purpose registers used during processing within the environment.These registers include information that represents the state of theenvironment at any particular point in time.

Moreover, native central processing unit 22 executes instructions andcode that are stored in memory 24. In one particular example, thecentral processing unit executes emulator code 32 stored in memory 24.This code enables the computing environment configured in onearchitecture to emulate another architecture. For instance, emulatorcode 32 allows machines based on architectures other than thez/Architecture, such as PowerPC processors, pSeries servers, or otherservers or processors, to emulate the z/Architecture and to executesoftware and instructions developed based on the z/Architecture.

Further details relating to emulator code 32 are described withreference to FIG. 32B. Guest instructions 40 stored in memory 24comprise software instructions (e.g., correlating to machineinstructions) that were developed to be executed in an architectureother than that of native CPU 22. For example, guest instructions 40 mayhave been designed to execute on a z/Architecture processor, butinstead, are being emulated on native CPU 22, which may be, for example,an Intel processor. In one example, emulator code 32 includes aninstruction fetching routine 42 to obtain one or more guest instructions40 from memory 24, and to optionally provide local buffering for theinstructions obtained. It also includes an instruction translationroutine 44 to determine the type of guest instruction that has beenobtained and to translate the guest instruction into one or morecorresponding native instructions 46. This translation includes, forinstance, identifying the function to be performed by the guestinstruction and choosing the native instruction(s) to perform thatfunction.

Further, emulator code 32 includes an emulation control routine 48 tocause the native instructions to be executed. Emulation control routine48 may cause native CPU 22 to execute a routine of native instructionsthat emulate one or more previously obtained guest instructions and, atthe conclusion of such execution, return control to the instructionfetch routine to emulate the obtaining of the next guest instruction ora group of guest instructions. Execution of native instructions 46 mayinclude loading data into a register from memory 24; storing data backto memory from a register; or performing some type of arithmetic orlogic operation, as determined by the translation routine.

Each routine is, for instance, implemented in software, which is storedin memory and executed by native central processing unit 22. In otherexamples, one or more of the routines or operations are implemented infirmware, hardware, software or some combination thereof. The registersof the emulated processor may be emulated using registers 30 of thenative CPU or by using locations in memory 24. In embodiments, guestinstructions 40, native instructions 46 and emulator code 32 may residein the same memory or may be disbursed among different memory devices.

As used herein, firmware includes, e.g., the microcode or Millicode ofthe processor. It includes, for instance, the hardware-levelinstructions and/or data structures used in implementation of higherlevel machine code. In one embodiment, it includes, for instance,proprietary code that is typically delivered as microcode that includestrusted software or microcode specific to the underlying hardware andcontrols operating system access to the system hardware.

A guest instruction 40 that is obtained, translated and executed may be,for instance, one of the instructions described herein. The instruction,which is of one architecture (e.g., the z/Architecture), is fetched frommemory, translated and represented as a sequence of native instructions46 of another architecture (e.g., PowerPC, pSeries, Intel, etc.). Thesenative instructions are then executed.

One or more aspects may relate to cloud computing.

It is understood in advance that although this disclosure includes adetailed description on cloud computing, implementation of the teachingsrecited herein are not limited to a cloud computing environment. Rather,embodiments of the present invention are capable of being implemented inconjunction with any other type of computing environment now known orlater developed.

Cloud computing is a model of service delivery for enabling convenient,on-demand network access to a shared pool of configurable computingresources (e.g. networks, network bandwidth, servers, processing,memory, storage, applications, virtual machines, and services) that canbe rapidly provisioned and released with minimal management effort orinteraction with a provider of the service. This cloud model may includeat least five characteristics, at least three service models, and atleast four deployment models.

Characteristics are as follows:

On-demand self-service: a cloud consumer can unilaterally provisioncomputing capabilities, such as server time and network storage, asneeded automatically without requiring human interaction with theservice's provider.

Broad network access: capabilities are available over a network andaccessed through standard mechanisms that promote use by heterogeneousthin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling: the provider's computing resources are pooled to servemultiple consumers using a multi-tenant model, with different physicaland virtual resources dynamically assigned and reassigned according todemand. There is a sense of location independence in that the consumergenerally has no control or knowledge over the exact location of theprovided resources but may be able to specify location at a higher levelof abstraction (e.g., country, state, or datacenter).

Rapid elasticity: capabilities can be rapidly and elasticallyprovisioned, in some cases automatically, to quickly scale out andrapidly released to quickly scale in. To the consumer, the capabilitiesavailable for provisioning often appear to be unlimited and can bepurchased in any quantity at any time.

Measured service: cloud systems automatically control and optimizeresource use by leveraging a metering capability at some level ofabstraction appropriate to the type of service (e.g., storage,processing, bandwidth, and active user accounts). Resource usage can bemonitored, controlled, and reported providing transparency for both theprovider and consumer of the utilized service.

Service Models are as follows:

Software as a Service (SaaS): the capability provided to the consumer isto use the provider's applications running on a cloud infrastructure.The applications are accessible from various client devices through athin client interface such as a web browser (e.g., web-based email). Theconsumer does not manage or control the underlying cloud infrastructureincluding network, servers, operating systems, storage, or evenindividual application capabilities, with the possible exception oflimited user-specific application configuration settings.

Platform as a Service (PaaS): the capability provided to the consumer isto deploy onto the cloud infrastructure consumer-created or acquiredapplications created using programming languages and tools supported bythe provider. The consumer does not manage or control the underlyingcloud infrastructure including networks, servers, operating systems, orstorage, but has control over the deployed applications and possiblyapplication hosting environment configurations.

Infrastructure as a Service (IaaS): the capability provided to theconsumer is to provision processing, storage, networks, and otherfundamental computing resources where the consumer is able to deploy andrun arbitrary software, which can include operating systems andapplications. The consumer does not manage or control the underlyingcloud infrastructure but has control over operating systems, storage,deployed applications, and possibly limited control of select networkingcomponents (e.g., host firewalls).

Deployment Models are as follows:

Private cloud: the cloud infrastructure is operated solely for anorganization. It may be managed by the organization or a third party andmay exist on-premises or off-premises.

Community cloud: the cloud infrastructure is shared by severalorganizations and supports a specific community that has shared concerns(e.g., mission, security requirements, policy, and complianceconsiderations). It may be managed by the organizations or a third partyand may exist on-premises or off-premises.

Public cloud: the cloud infrastructure is made available to the generalpublic or a large industry group and is owned by an organization sellingcloud services.

Hybrid cloud: the cloud infrastructure is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together by standardized or proprietary technology thatenables data and application portability (e.g., cloud bursting forloadbalancing between clouds).

A cloud computing environment is service oriented with a focus onstatelessness, low coupling, modularity, and semantic interoperability.At the heart of cloud computing is an infrastructure comprising anetwork of interconnected nodes.

Referring now to FIG. 33, illustrative cloud computing environment 50 isdepicted. As shown, cloud computing environment 50 comprises one or morecloud computing nodes 10 with which local computing devices used bycloud consumers, such as, for example, personal digital assistant (PDA)or cellular telephone 54A, desktop computer 54B, laptop computer 54C,and/or automobile computer system 54N may communicate. Nodes 10 maycommunicate with one another. They may be grouped (not shown) physicallyor virtually, in one or more networks, such as Private, Community,Public, or Hybrid clouds as described hereinabove, or a combinationthereof. This allows cloud computing environment 50 to offerinfrastructure, platforms and/or software as services for which a cloudconsumer does not need to maintain resources on a local computingdevice. It is understood that the types of computing devices 54A-N shownin FIG. 33 are intended to be illustrative only and that computing nodes10 and cloud computing environment 50 can communicate with any type ofcomputerized device over any type of network and/or network addressableconnection (e.g., using a web browser).

Referring now to FIG. 34, a set of functional abstraction layersprovided by cloud computing environment 50 (FIG. 33) is shown. It shouldbe understood in advance that the components, layers, and functionsshown in FIG. 34 are intended to be illustrative only and embodiments ofthe invention are not limited thereto. As depicted, the following layersand corresponding functions are provided:

Hardware and software layer 60 includes hardware and softwarecomponents. Examples of hardware components include mainframes 61; RISC(Reduced Instruction Set Computer) architecture based servers 62;servers 63; blade servers 64; storage devices 65; and networks andnetworking components 66. In some embodiments, software componentsinclude network application server software 67 and database software 68.

Virtualization layer 70 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers71; virtual storage 72; virtual networks 73, including virtual privatenetworks; virtual applications and operating systems 74; and virtualclients 75.

In one example, management layer 80 may provide the functions describedbelow. Resource provisioning 81 provides dynamic procurement ofcomputing resources and other resources that are utilized to performtasks within the cloud computing environment. Metering and Pricing 82provide cost tracking as resources are utilized within the cloudcomputing environment, and billing or invoicing for consumption of theseresources. In one example, these resources may comprise applicationsoftware licenses. Security provides identity verification for cloudconsumers and tasks, as well as protection for data and other resources.User portal 83 provides access to the cloud computing environment forconsumers and system administrators. Service level management 84provides cloud computing resource allocation and management such thatrequired service levels are met. Service Level Agreement (SLA) planningand fulfillment 85 provide pre-arrangement for, and procurement of,cloud computing resources for which a future requirement is anticipatedin accordance with an SLA.

Workloads layer 90 provides examples of functionality for which thecloud computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation 91; software development and lifecycle management 92; virtualclassroom education delivery 93; data analytics processing 94;transaction processing 95; and table of contents processing 96.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

In addition to the above, one or more aspects may be provided, offered,deployed, managed, serviced, etc. by a service provider who offersmanagement of customer environments. For instance, the service providercan create, maintain, support, etc. computer code and/or a computerinfrastructure that performs one or more aspects for one or morecustomers. In return, the service provider may receive payment from thecustomer under a subscription and/or fee agreement, as examples.Additionally or alternatively, the service provider may receive paymentfrom the sale of advertising content to one or more third parties.

In one aspect, an application may be deployed for performing one or moreembodiments. As one example, the deploying of an application comprisesproviding computer infrastructure operable to perform one or moreembodiments.

As a further aspect, a computing infrastructure may be deployedcomprising integrating computer readable code into a computing system,in which the code in combination with the computing system is capable ofperforming one or more embodiments.

As yet a further aspect, a process for integrating computinginfrastructure comprising integrating computer readable code into acomputer system may be provided. The computer system comprises acomputer readable medium, in which the computer medium comprises one ormore embodiments. The code in combination with the computer system iscapable of performing one or more embodiments.

Although various embodiments are described above, these are onlyexamples. For example, computing environments of other architectures canbe used to incorporate and use one or more embodiments. Further,different instructions or operations may be used. Additionally,different registers may be used and/or other types of indications (otherthan register numbers) may be specified. Many variations are possible.

Further, other types of computing environments can benefit and be used.As an example, a data processing system suitable for storing and/orexecuting program code is usable that includes at least two processorscoupled directly or indirectly to memory elements through a system bus.The memory elements include, for instance, local memory employed duringactual execution of the program code, bulk storage, and cache memorywhich provide temporary storage of at least some program code in orderto reduce the number of times code must be retrieved from bulk storageduring execution.

Input/Output or I/O devices (including, but not limited to, keyboards,displays, pointing devices, DASD, tape, CDs, DVDs, thumb drives andother memory media, etc.) can be coupled to the system either directlyor through intervening I/O controllers. Network adapters may also becoupled to the system to enable the data processing system to becomecoupled to other data processing systems or remote printers or storagedevices through intervening private or public networks. Modems, cablemodems, and Ethernet cards are just a few of the available types ofnetwork adapters.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting. As used herein, thesingular forms “a”, “an” and “the” are intended to include the pluralforms as well, unless the context clearly indicates otherwise. It willbe further understood that the terms “comprises” and/or “comprising”,when used in this specification, specify the presence of statedfeatures, integers, steps, operations, elements, and/or components, butdo not preclude the presence or addition of one or more other features,integers, steps, operations, elements, components and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below, if any, areintended to include any structure, material, or act for performing thefunction in combination with other claimed elements as specificallyclaimed. The description of one or more embodiments has been presentedfor purposes of illustration and description, but is not intended to beexhaustive or limited to in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the art. Theembodiment was chosen and described in order to best explain variousaspects and the practical application, and to enable others of ordinaryskill in the art to understand various embodiments with variousmodifications as are suited to the particular use contemplated.

What is claimed is:
 1. A computer program product for facilitatingprocessing within a computing environment, the computer program productcomprising: a non-transitory computer readable storage medium readableby a processing circuit and storing instructions for performing a methodcomprising: determining that a unit of memory that provides storage forone or more in-memory configuration state registers is to be pinned in asystem that includes a guest operating system and one or more hosts atone or more host levels; notifying, based on determining that the unitof memory is to be pinned, at least one host of the one or more hoststhat the unit of memory is to be pinned; determining that the unit ofmemory is to be unpinned; and resetting an indicator used to indicatepinning, based on determining that the unit of memory is to be unpinned.2. The computer program product of claim 1, wherein the determining isperformed based on receiving a new configuration value for the unit ofmemory.
 3. The computer program product of claim 2, wherein the newconfiguration value comprises a guest address used to locate the unit ofmemory.
 4. The computer program product of claim 3, wherein the methodfurther comprises: selecting a host level of the one or more hostlevels; translating the guest address to a host address for the selectedhost level; and notifying the selected host level of the pinning.
 5. Thecomputer program product of claim 4, wherein the method furthercomprises repeating the selecting, the translating and the notifying foreach host level of the one or more host levels.
 6. The computer programproduct of claim 4, wherein the notifying comprises setting an indicatorin a page table entry corresponding to the selected host level.
 7. Thecomputer program product of claim 1, wherein the notifying comprisessetting an indicator in a page table entry corresponding to the unit ofmemory.
 8. The computer program product of claim 1, wherein thenotifying comprises raising an interrupt.
 9. The computer programproduct of claim 1, wherein the unit of memory is a page of memory. 10.A computer system for facilitating processing within a computingenvironment, the computer system comprising: a memory; and a processorin communication with the memory, wherein the computer system isconfigured to perform a method, said method comprising: determining thata unit of memory that provides storage for one or more in-memoryconfiguration state registers is to be pinned in a system that includesa guest operating system and one or more hosts at one or more hostlevels; notifying, based on determining that the unit of memory is to bepinned, at least one host of the one or more hosts that the unit ofmemory is to be pinned; determining that the unit of memory is to beunpinned; and resetting an indicator used to indicate pinning, based ondetermining that the unit of memory is to be unpinned.
 11. The computersystem of claim 10, wherein the determining is performed based onreceiving a new configuration value for the unit of memory.
 12. Thecomputer system of claim 11, wherein the new configuration valuecomprises a guest address used to locate the unit of memory.
 13. Thecomputer system of claim 12, wherein the method further comprises:selecting a host level of the one or more host levels; translating theguest address to a host address for the selected host level; andnotifying the selected host level of the pinning.
 14. The computersystem of claim 10, wherein the notifying comprises setting an indicatorin a page table entry corresponding to the unit of memory.
 15. Thecomputer system of claim 10, wherein the notifying comprises raising aninterrupt.
 16. A computer-implemented method of facilitating processingwithin a computing environment, the computer-implemented methodcomprising: determining that a unit of memory that provides storage forone or more in-memory configuration state registers is to be pinned in asystem that includes a guest operating system and one or more hosts atone or more host levels; notifying, based on determining that the unitof memory is to be pinned, at least one host of the one or more hoststhat the unit of memory is to be pinned: determining that the unit ofmemory is to be unpinned; and resetting an indicator used to indicatepinning, based on determining that the unit of memory is to be unpinned.17. The computer-implemented method of claim 16, wherein the determiningis performed based on receiving a new configuration value for the unitof memory.
 18. The computer-implemented method of claim 17, wherein thenew configuration value comprises a guest address used to locate theunit of memory.
 19. The computer-implemented method of claim 18, furthercomprising: selecting a host level of the one or more host levels;translating the guest address to a host address for the selected hostlevel; and notifying the selected host level of the pinning.
 20. Thecomputer-implemented method of claim 16, wherein the notifying comprisessetting an indicator in a page table entry corresponding to the unit ofmemory.